feat(tools/skill-and-tool-validator): add SOFT SKILL.md line-limit check#756
Merged
Merged
Conversation
Adds check apache#20: a SOFT advisory flagging any SKILL.md entrypoint over SKILL_LINE_LIMIT (500) lines, per PRINCIPLES.md §14. Advisory-only, so pre-existing over-limit skills are surfaced for gradual migration without blocking unrelated work; new oversized entrypoints are caught. Rebased onto latest main as a single clean commit (the branch had picked up already-merged fossil/vcs history); test_under_limit now uses a genuinely under-limit count rather than duplicating the at-limit boundary test. Generated-by: Claude Code (Opus 4.8)
2086caa to
81528a7
Compare
potiuk
approved these changes
Jul 5, 2026
potiuk
left a comment
Member
There was a problem hiding this comment.
Nice check — a clean, well-scoped SOFT advisory that stops new oversized SKILL.md entrypoints from slipping in, while leaving the existing over-limit skills as advisories to migrate deliberately rather than blocking unrelated work. The implementation is tidy (SKILL.md-only guard, clear message pointing at PRINCIPLE 14) and the test coverage is thorough.
A couple of things I did before merging (you're preserved as commit author):
- Rebased onto latest main as a single clean commit. The branch had picked up a good chunk of already-merged history (fossil / vcs /
uv.lock), so a plaingit rebasereplayed all of that and conflicted. The net two-file change applied cleanly to current main via a 3-way apply, so the check is confirmed to still work against today's tree. - Removed the duplicate test.
test_under_limit_no_violationwas asserting on exactlySKILL_LINE_LIMIT(500), identical totest_at_limit_no_violation— it now uses a genuinely under-limit count (SKILL_LINE_LIMIT - 100) so "under" and "at limit" are distinct cases.
Ran the full validator pytest suite + the validator itself locally — all green, and the new advisory fires on the 21 real over-limit skills as expected. Merging. Thanks!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds check #20 to the skill-and-tool-validator: a SOFT advisory that flags
any
SKILL.mdentrypoint exceedingSKILL_LINE_LIMIT(500 lines), perPRINCIPLES.md §14.
Why
Oversized SKILL.md entrypoints should move reference material into sibling
markdown files linked one level deep. This check keeps new oversized
entrypoints from being merged unnoticed, while staying advisory so existing
over-limit skills can be migrated deliberately rather than blocking unrelated
work.
Behaviour
skill-line-limit, registered as SOFT: surfaces as a stderradvisory and does not fail the run unless
--strictis passed.SKILL.md.Type of change
.claude/skills/<name>/) — eval fixtures updated belowtools/<system>/*.md)tools/*/withpyproject.toml)docs/,README.md,CONTRIBUTING.md)projects/_template/)prek, workflows, validators)Test plan
prek run --all-filespassesuv run pytest/ruff check/mypypasses(
PYTHONPATH=tools/skill-evals/src python3 -m skill_evals.runner tools/skill-evals/evals/<skill>/)(a regression test for the bug fixed / the behaviour added — see CONTRIBUTING.md)