Skip to content

feat(tools/skill-and-tool-validator): add SOFT SKILL.md line-limit check#756

Merged
potiuk merged 1 commit into
apache:mainfrom
justinmclean:skill-md-line-limit-check
Jul 5, 2026
Merged

feat(tools/skill-and-tool-validator): add SOFT SKILL.md line-limit check#756
potiuk merged 1 commit into
apache:mainfrom
justinmclean:skill-md-line-limit-check

Conversation

@justinmclean

Copy link
Copy Markdown
Member

Summary

Adds check #20 to the skill-and-tool-validator: a SOFT advisory that flags
any SKILL.md entrypoint exceeding SKILL_LINE_LIMIT (500 lines), per
PRINCIPLES.md §14.

Why

Oversized SKILL.md entrypoints should move reference material into sibling
markdown files linked one level deep. This check keeps new oversized
entrypoints from being merged unnoticed, while staying advisory so existing
over-limit skills can be migrated deliberately rather than blocking unrelated
work.

Behaviour

  • Category skill-line-limit, registered as SOFT: surfaces as a stderr
    advisory and does not fail the run unless --strict is passed.
  • Only applies to files named SKILL.md.
  • Message points to PRINCIPLES.md §14 and names the actual line count.
  • Currently flags 19 pre-existing over-limit skills as advisories.

Type of change

  • Skill change (.claude/skills/<name>/) — eval fixtures updated below
  • Tool / bridge contract (tools/<system>/*.md)
  • Python package (tools/*/ with pyproject.toml)
  • Groovy reference impl
  • Cross-cutting (RFC, AGENTS.md, sandbox, privacy-LLM)
  • Documentation (docs/, README.md, CONTRIBUTING.md)
  • Project template (projects/_template/)
  • CI / dev loop (prek, workflows, validators)
  • Other:

Test plan

  • [X ] prek run --all-files passes
  • For Python packages touched: uv run pytest / ruff check / mypy passes
  • For Groovy bridges touched: command-line invocation tested end-to-end
  • For skill changes: eval suite passes for the affected skill
    (PYTHONPATH=tools/skill-evals/src python3 -m skill_evals.runner tools/skill-evals/evals/<skill>/)
  • For skill behaviour changes: a new or updated eval fixture is included in this PR
    (a regression test for the bug fixed / the behaviour added — see CONTRIBUTING.md)
  • Other:

@justinmclean justinmclean self-assigned this Jul 5, 2026
Adds check apache#20: a SOFT advisory flagging any SKILL.md entrypoint over
SKILL_LINE_LIMIT (500) lines, per PRINCIPLES.md §14. Advisory-only, so
pre-existing over-limit skills are surfaced for gradual migration without
blocking unrelated work; new oversized entrypoints are caught.

Rebased onto latest main as a single clean commit (the branch had picked up
already-merged fossil/vcs history); test_under_limit now uses a genuinely
under-limit count rather than duplicating the at-limit boundary test.

Generated-by: Claude Code (Opus 4.8)
@potiuk potiuk force-pushed the skill-md-line-limit-check branch from 2086caa to 81528a7 Compare July 5, 2026 19:23

@potiuk potiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice check — a clean, well-scoped SOFT advisory that stops new oversized SKILL.md entrypoints from slipping in, while leaving the existing over-limit skills as advisories to migrate deliberately rather than blocking unrelated work. The implementation is tidy (SKILL.md-only guard, clear message pointing at PRINCIPLE 14) and the test coverage is thorough.

A couple of things I did before merging (you're preserved as commit author):

  • Rebased onto latest main as a single clean commit. The branch had picked up a good chunk of already-merged history (fossil / vcs / uv.lock), so a plain git rebase replayed all of that and conflicted. The net two-file change applied cleanly to current main via a 3-way apply, so the check is confirmed to still work against today's tree.
  • Removed the duplicate test. test_under_limit_no_violation was asserting on exactly SKILL_LINE_LIMIT (500), identical to test_at_limit_no_violation — it now uses a genuinely under-limit count (SKILL_LINE_LIMIT - 100) so "under" and "at limit" are distinct cases.

Ran the full validator pytest suite + the validator itself locally — all green, and the new advisory fires on the 21 real over-limit skills as expected. Merging. Thanks!

@potiuk potiuk merged commit 27e7c5e into apache:main Jul 5, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants