gather: deterministic context-pack tool (aft_gather)

[iceteaSA]

gather: deterministic context-pack tool (aft_gather)

What

One new tool — aft_gather — assembles a bounded "context pack" (ranked, deduped, budgeted verbatim code evidence) in a single call. It replaces the multi-turn search → outline → zoom → callgraph read chain an agent otherwise runs to build context around a question or a symbol.

Two modes (mutually exclusive):

• question: "how does X work?" — seeds from handle_semantic_search (same pipeline as aft_search, all lanes/fallbacks)
• symbol + filePath — seeds from the callgraph (impact depth-1 callers + call_tree depth-1 callees)

Seeds expand one hop through the callgraph, dedupe by canonicalized (file, symbol) with seeds winning, and render via render_symbol_within_budget until a hard line budget (default 400, cap 800) is spent. Everything past the cut appears as one-line stubs under ## Beyond budget (zoom to expand) — nothing is silently dropped.

Why

Agents burn serial turns assembling context: search, then outline the hits, then zoom the symbols, then chase callers. Each turn round-trips through the model. A pack returns evidence (verbatim bodies with file:line headers), not conclusions — the agent reasons over it directly, ready to attach to a subagent dispatch.

Measured on a real config repo (3 questions, one call each vs. the manual chain):

• "how does the lane-verdict cache flow end-to-end" → complete 4-file chain (cache shapes, verdict logic, write/read, plugin deny hook) in one pack, used=283/400. Manual baseline: 5-6 tool calls.
• "how does score-tap decide when to write an executor row" → the full decision chain (event filter → dedup → context sources → verdict parse → DB insert) in one pack at budget=200.
• An unrehearsed cross-file question → correct core symbols plus their docstrings, first try.

Independently reproduced on the aft codebase itself: question: "how does bash output compression dispatch pick a compressor" → seeds=15, used=226/400, one pack assembling the full dispatch chain (compress → gate → compress_with_registry_exit_code 20-compressor array → Compressor trait → install path → subc mirror) that otherwise takes a 4–5-call search→zoom chain.

Honest degradation

The pack never lies about its own quality:

• While the semantic index is building, long NL queries degrade to lexical-only file-level hits. The pack renders them as visible file:line (no containing symbol) stubs and flags the header with degraded=semantic-index-building (partial results — retry when index ready) — detected via the response's semantic_status field, cleared as soon as one real seed resolves. No blocking or retry inside the tool.
• Grep-fallback hits ({file, line_text, line}, no symbol name) resolve to their containing symbol by line containment — definitions, call sites, and comment hits all upgrade to the enclosing symbol. Hits with no containing symbol stay visible as stubs.
• Unresolved external/stdlib callees collapse to one summary line ((N unresolved external calls omitted)) instead of drowning the stub list; unresolved seeds and callers are never suppressed.

Implementation

• crates/aft/src/commands/gather.rs — Rust-side composition: calls handle_semantic_search / impact_result / call_tree_result / render_symbol_within_budget directly (shared &AppContext, no bridge round-trips, no parallel reimplementation of search).
• Wiring: main.rs dispatch arm, subc_translate.rs mapping, TS factory packages/opencode-plugin/src/tools/gather.ts + registration (same tier as aft_callgraph — depends on the callgraph store).
• No new dependencies, no config surface beyond the tool args, no LLM calls, no caching.
• Follows the tri-state honest-reporting convention (protocol.rs Response doc-comment): success:false+code for un-performable calls (e.g. invalid_request on a bad mode combo), success:true with a visible degraded/stub pack for partial results — never a bare empty success.

Tests

22 unit tests in gather.rs, including red-checked regressions (each confirmed to fail against pre-fix code): mid-codepoint truncation panic, duplicate-symbol line-anchored resolution, abs/rel path dedupe, containing-symbol resolution via a real TreeSitterProvider, callee-only stub suppression driven through the production build_pack path, and degradation-flag presence/absence/mixed cases.

Limitations (deliberate scope)

• 1-hop expansion only — multi-hop was deliberately excluded: the budget math and ranking get harder and the packs get noisier.
• Callgraph-dependent by design: neighbor expansion quality follows the callgraph store's freshness, same as aft_callgraph.
• Budget counted in lines, not tokens — matches aft's existing budget idiom across zoom/outline.

---

^{Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.}

---

Summary by cubic

Adds aft_gather, a one-call, deterministic context-pack builder that returns ranked, deduped, verbatim code within a fixed line budget, replacing the multi-step search→outline→zoom→callgraph chain. Exposed as gather on Rust and as aft_gather in @opencode (tier: “all”).

• New Features

  • Two modes: question (semantic seeds) or symbol+filePath (impact callers + call-tree callees); 1‑hop expansion; dedupe by normalized (file, symbol) with seeds winning.
  • Hard line budget (default 400, max 800) via render_symbol_within_budget; overflow listed under “Beyond budget”; unresolved external callees collapse to one summary line; seeds/callers stay visible as stubs.
  • Grep‑fallback hits resolve to containing symbols; hits with no containing symbol remain as visible stubs.
  • Header flags semantic-index-building degradation and neighbors=skipped(callgraph-unavailable) when the callgraph store is down.
  • Wired into Rust dispatch (gather) with translate_gather (validates modes, resolves filePath); tool added in @opencode with arg validation (budget 1–800). No new dependencies. 22 tests.

• Migration

  • Call with either { question: "how does X work?" } or { symbol: "foo", filePath: "src/foo.rs" }; optional budget 1–800 (default 400).
  • Neighbor expansion uses the callgraph store; required in symbol mode. In question mode, neighbors are skipped with a header notice if unavailable.

^{Written for commit 1d068c9. Summary will update on new commits.}

Greptile Summary

This PR adds aft_gather, a single-call "context pack" builder that replaces the search → outline → zoom → callgraph chain an agent would otherwise run to collect code evidence around a question or symbol. The implementation is self-contained, re-uses existing pipeline functions (handle_semantic_search, impact_result, call_tree_result, render_symbol_within_budget), and adds no new dependencies.

• Core logic (gather.rs): Two mutually-exclusive modes (question-seeded via semantic search, or symbol-seeded via impact + call-tree); 1-hop neighbor expansion; dedup by normalized (file, symbol) key with seeds winning; hard line budget with overflow stubs; explicit degradation flags (semantic-index-building, callgraph-unavailable); external-callee-only suppression for unresolvable stdlib symbols. Previously-flagged issues (dead zero-branch, budget undercounting by separator lines, brittle used= in-place patch) are all confirmed fixed.
• Wiring: main.rs dispatch arm, subc_translate.rs translate_gather with three-way validation, TypeScript tool definition in gather.ts registered under the "all" tier alongside aft_callgraph.
• Tests: 22 unit tests including UTF-8 truncation regression, abs/rel path dedup, containing-symbol resolution via a real TreeSitterProvider, callee-only suppression through the production build_pack path, and degradation-flag presence/absence/mixed cases.

Confidence Score: 5/5

Safe to merge; all three previously-flagged issues (dead branch, budget undercounting, brittle in-place patch) are confirmed resolved in the current HEAD, and no new correctness defects were found.

The implementation correctly reuses existing pipeline functions, the callgraph store's path normalization handles relative paths so question-mode neighbor expansion works as documented, budget accounting is exact (separator lines counted), and the 22 tests cover the key regression scenarios including the UTF-8 truncation panic and callee-suppression scoping.

No files require special attention; the only findings are doc-comment and error-message polish in gather.rs.

Important Files Changed

Filename	Overview
crates/aft/src/commands/gather.rs	New 1460-line implementation of aft_gather: two-mode context-pack builder with semantic-search and callgraph seeding, 1-hop neighbor expansion, per-symbol budget rendering, dedup, degradation flagging, and 22 unit tests. Previously-flagged issues (dead branch, budget undercounting, brittle used= patch) are all resolved per the PR thread.
packages/opencode-plugin/src/tools/gather.ts	TypeScript tool definition and execute handler for aft_gather. Validates mode combinations with clear error messages and delegates to the Rust gather command via callToolCall.
packages/opencode-plugin/src/index.ts	Registers gatherTools in the plugin's tool map and adds aft_gather to the ALL_ONLY_TOOLS set, restricting it to the "all" tier alongside aft_callgraph.
crates/aft/src/subc_translate.rs	Adds translate_gather with three-way validation (conflicting modes, unpaired symbol/filePath, neither mode) and filePath resolution against project root — more precise error messages than the Rust-side handle_gather.
crates/aft/src/main.rs	Adds "gather" dispatch arm routing to handle_gather; single-line change alongside other callgraph-tier commands.
crates/aft/src/commands/mod.rs	Adds pub mod gather; no functional change.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[aft_gather call] --> B{Mode?}
    B -- question --> C[handle_semantic_search\ntop_k=15]
    B -- symbol + filePath --> D[validate_path]
    C --> E{seeds empty?}
    E -- yes, no stubs --> F[Return: no results]
    E -- yes, has stubs --> G[build_pack\nseeds=empty\npre_stubs=visible]
    E -- no --> H[collect_callgraph_neighbors\n1-hop callers + callees]
    D --> I[impact_result\ndepth=1 callers]
    I --> J[collect_callees_for_seed\ncall_tree depth=1]
    J --> K[build_pack]
    H --> K
    K --> L[Dedup by file+name\nseeds win]
    L --> M[Per-symbol budget render\nrender_symbol_within_budget]
    M --> N{fits budget?}
    N -- yes --> O[Append rendered body\nlines_used += section_lines + 1]
    N -- no --> P[Add to stubs list]
    O --> Q{more candidates?}
    P --> Q
    Q -- yes --> M
    Q -- no --> R[Build header with final used=N]
    R --> S[Append Beyond budget stubs\n+ unresolved callee summary]
    S --> T[Response success text]

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[aft_gather call] --> B{Mode?}
    B -- question --> C[handle_semantic_search\ntop_k=15]
    B -- symbol + filePath --> D[validate_path]
    C --> E{seeds empty?}
    E -- yes, no stubs --> F[Return: no results]
    E -- yes, has stubs --> G[build_pack\nseeds=empty\npre_stubs=visible]
    E -- no --> H[collect_callgraph_neighbors\n1-hop callers + callees]
    D --> I[impact_result\ndepth=1 callers]
    I --> J[collect_callees_for_seed\ncall_tree depth=1]
    J --> K[build_pack]
    H --> K
    K --> L[Dedup by file+name\nseeds win]
    L --> M[Per-symbol budget render\nrender_symbol_within_budget]
    M --> N{fits budget?}
    N -- yes --> O[Append rendered body\nlines_used += section_lines + 1]
    N -- no --> P[Add to stubs list]
    O --> Q{more candidates?}
    P --> Q
    Q -- yes --> M
    Q -- no --> R[Build header with final used=N]
    R --> S[Append Beyond budget stubs\n+ unresolved callee summary]
    S --> T[Response success text]

Loading

_{Reviews (4): Last reviewed commit: "gather: deterministic context-pack tool ..." | Re-trigger Greptile}

[cubic-dev-ai]

4 issues found across 7 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[iceteaSA]

Follow-up on the two maintainability notes from the Greptile summary (they weren't separate review threads, so noting here) — both addressed in cf09b9b9:

• degraded expression flagged as always-false in the non-empty-seeds path — intentional: degraded only applies when zero symbol-level seeds resolve (a building index yields no seeds; a ready index with seeds is never degraded). Logic unchanged; added a comment on the why so it doesn't read as dead code.
• callee-suppression string-match coupling — extracted UNRESOLVED_MARKER ("(symbol not resolved)") and CALLEE_PROVENANCE_PREFIX ("callee-of-") consts, referenced by both the producer (render_symbol_section err arm / collect_callees_for_seed) and the consumer (build_pack suppression guard) so the match can't silently drift. Only test-site literals remain, intentionally — a test pointing at the const couldn't catch const drift.

Matched/rendered text is byte-identical; 23/23 gather tests green.