feat(providers): local agent-CLI providers (claude/codex/gemini), no API key

[AbhiramDwivedi]

Closes #57

What

Local agent-CLI providers that route Stage-2 LLM analysis through a locally-installed, already-authenticated agent CLI instead of a metered HTTP API — no API key needed, the CLI's own login session is used. Supported today: claude, codex, gemini (via SKILLSPECTOR_PROVIDER=claude_cli / codex_cli / gemini_cli). Adding another CLI is a small registry entry + a ~5-line provider subclass — see the "HOW TO ADD A NEW AGENT CLI" guide in providers/_agent_cli.py.

How

• One hardened chokepoint (providers/_agent_cli.py): shell=False; untrusted prompt via stdin only, never argv; capability stripping verified against the real CLIs — claude: --allowed-tools "" deny-by-default + --permission-mode dontAsk + --strict-mcp-config + --disable-slash-commands; codex: --sandbox read-only + --ephemeral + --ignore-user-config; gemini: --approval-mode plan (read-only, no tool execution). --dangerously-skip-permissions / auto-approve is never used. Scrubbed env; temp CWD; per-call timeout; streamed stdout with a hard size cap (process killed on overflow — no unbounded buffering); model-label validated against argument injection; fail-closed on every error path. A per-CLI CliSpec registry keeps all CLI-specific argv/parse/auth behind one lookup, so the shared security core is unchanged when a CLI is added.
• The analyzer seam is untouched. LLM analyzers get their model from get_chat_model(); for CLI providers that returns a minimal ChatOpenAI-compatible adapter backed by provider.complete(), with structured output via prompt-for-JSON + Pydantic validation (fail-closed). HTTP providers are unchanged.
• No pinned model versions. A CLI provider forwards no --model by default, so it runs with the user's OWN configured model and thinking level; set SKILLSPECTOR_MODEL to override. (No bundled model_registry.yaml; CLI providers use the package-wide default token budgets.)
• is_available() does a real local auth probe (claude auth status / codex login status) so a report's llm_available never claims availability when the CLI is logged out.

Antigravity (agy) — registered but disabled

antigravity_cli is wired into the registry but fail-closed and disabled. Tested end-to-end against the real agy: its --print mode renders to a TTY and returns empty stdout on a pipe (how the runner must capture it), and it takes the prompt as an argv value rather than stdin — so it cannot be driven programmatically. Its backend is Gemini, so use gemini_cli for that capability. The registry entry documents the finding in one place; enabling it later is a one-function change if agy gains a headless stdout mode.

Test

Fully-mocked unit tests (no CLI required) cover the subprocess invariants, the bounded reader (real subprocesses: normal / overflow-kill / timeout), the adapter + structured-output parsing, provider selection, the registry, no-pinned-model resolution, and the disabled-antigravity guard — so a contributor without any of these CLIs installed runs the entire default suite. An opt-in live harness (tests/integration/test_agent_cli_live.py, marked integration, excluded by default) is parametrized over claude/codex/gemini and skips per-CLI when a binary is absent/unauthenticated. Verified end-to-end: claude/codex/gemini each return real output with no pinned model.

🤖 Generated with Claude Code

[AbhiramDwivedi]

Rebased onto the latest main (a5092dd) and adapted onto the recent provider refactor (the new providers/chat_models.py + create_chat_model() seam), so this is mergeable again. Collapsed to a single commit.

Integration summary

• HTTP providers now flow through your create_chat_model() (native ChatAnthropic, etc.); the CLI providers branch to a small AgentCLIChatModel adapter selected by a duck-typed has_cli_capability() check — the existing HTTP path is unchanged.
• is_llm_available() delegates to the provider's is_available() for CLI providers and falls back to credential resolution for HTTP providers.
• No model pinned: CLI providers forward no --model (the user's own CLI-configured model is used); SKILLSPECTOR_MODEL overrides.

Re-verified end-to-end after the rebase: claude, codex, and gemini each return a live response through the full chat_completion → AgentCLIChatModel → CLI path (both plain and structured-output), with no model pinned. Unit suite + ruff green; the hardened subprocess chokepoint (providers/_agent_cli.py) is unchanged from the pre-rebase commit.

[486]

claude_cli provider: _parse_claude_output rejects the array-shaped --output-format json emitted by some Claude Code builds

First, thanks for this PR — the hardened _agent_cli.py chokepoint (stdin-only untrusted content, capability stripping, env scrub, temp CWD, fail-closed) is exactly what's needed to point an LLM at adversarial skill content. Tried it end-to-end with SKILLSPECTOR_PROVIDER=claude_cli and hit one CLI-version compatibility issue worth flagging.

Symptom

With a Claude Code CLI at 2.1.178, every Stage-2 semantic analyzer fails and the scan silently degrades to static-only:

WARNING semantic_security_discovery failed: expected a JSON object from claude, got list: '[{"type":"system","subtype":"init", ...
WARNING semantic_developer_intent  failed: expected a JSON object from claude, got list: ...
WARNING semantic_quality_policy    failed: expected a JSON object from claude, got list: ...

Root cause

On this build, claude -p --output-format json returns a JSON array of stream events rather than a single result object. The assistant text lives in the final element (type == "result"):

$ printf 'Reply OK.' | claude -p --output-format json --allowed-tools "" \
    --permission-mode dontAsk --strict-mcp-config --disable-slash-commands \
  | python3 -c "import json,sys; d=json.load(sys.stdin); \
      print(type(d).__name__, [x.get('type') for x in d])"
list ['system', 'system', 'system', 'system', 'system', 'system', 'system', 'assistant', 'assistant', 'result']

_parse_claude_output only handles the single-dict envelope, so it raises expected a JSON object from claude, got list. The analyzers catch this and fall back to static-only, which is easy to miss unless you check the warnings.

Suggested fix — accept both shapes in _parse_claude_output: if the parsed JSON is a list, select the last element carrying a result key (preferring type == "result"), then continue with the existing dict handling:

    if isinstance(envelope, list):
        result_event = None
        for item in envelope:
            if isinstance(item, dict) and "result" in item:
                if str(item.get("type", "")).lower() == "result":
                    result_event = item
                elif result_event is None:
                    result_event = item
        if result_event is None:
            raise AgentCLIError(
                f"claude event array has no element with a 'result' key; "
                f"types={[i.get('type') for i in envelope if isinstance(i, dict)]!r}"
            )
        envelope = result_event

After this change the semantic analyzers run and metadata.llm_available == true, verified against a batch of real skills.

It may also be worth surfacing a partial-degradation signal (e.g. flip llm_available to false or warn loudly at the end) when the LLM stage was requested but every analyzer failed, so a parser/transport break can't quietly turn a deep scan into a static-only one.

Happy to open a PR with the parser change + a unit test for the array shape if useful.