Skip to content

ci: improve AI issue triager routing and prompts#252

Open
ayeshurun wants to merge 1 commit into
mainfrom
ayeshurun/dev-alonyeshurun-improve-ai-triager
Open

ci: improve AI issue triager routing and prompts#252
ayeshurun wants to merge 1 commit into
mainfrom
ayeshurun/dev-alonyeshurun-improve-ai-triager

Conversation

@ayeshurun

Copy link
Copy Markdown
Collaborator

Summary

Fixes several gaps found while reviewing recent triaged issues, triager runs, and prompt outputs. All changes are scoped to .github/ triage infra (workflow + prompt YAMLs); no CLI source or tests are touched.

Changes

1. Fixed the "documentation" black hole

  • New .github/prompts/documentation-triage.prompt.yml and a documentation entry in the workflow's labels_to_prompts_mapping. Documentation issues previously had no prompt mapping, so they kept needs triage forever with no AI comment. Its categories (Answered, Help Wanted, Needs Author Feedback, Needs Team Review) route through the existing post-processing keywords/labels — no new labels required.

2. Repaired the broken author-feedback loop

  • Needs Author Feedback / Requires Additional Details now applies the needs author feedback label (mirroring the /needinfo command), so resourceManagement.yml automatically re-triages when the author replies. Previously the loop never fired because the AI path never added that label.

3. Fixed @{issue_author} never being substituted

  • The "Resolve issue details" step prepends Issue author: @handle to the AI input, and all prompts now substitute that handle for {issue_author} instead of emitting the literal placeholder. Because the handle is prepended ahead of any [RE-TRIAGE] marker, re-triage detection changed from "starts with" to "contains".

4. Removed literal backticks around the Next Steps and footer lines in all prompts, so they render as markdown bold/blockquote rather than inline code.

5. Closed post-processing gaps

  • Valuable Enhancement → adds ai:needs team attention for backlog filtering (no @mention spam).
  • Out of Scope → auto-closes as not_planned (via a new closeReason).
  • Removed the dead needs maintainer input keyword and updated the triage summary table to match.

Verification

  • All prompt YAMLs, the workflow, and the policy file parse cleanly (yaml.safe_load).
  • Routing re-checked end-to-end: every new/changed assessment category maps to an existing post-processing branch and label.

Note

The branch name is ayeshurun/dev-alonyeshurun-improve-ai-triager rather than the requested dev/alonyeshurun/improve-ai-triager — the rename tooling slugifies slashes and namespaces under the handle. Happy to recreate the branch if the exact format is required.

Fix several gaps in the AI issue triage workflow and prompts:

- Add a documentation-triage prompt and map the 'documentation' label
  to it. Documentation issues previously had no prompt, so they kept
  'needs triage' forever with no AI comment.
- Repair the author-feedback loop: 'Needs Author Feedback' /
  'Requires Additional Details' now applies the 'needs author feedback'
  label, mirroring '/needinfo' so the policy automation re-triages when
  the author replies.
- Substitute the real author handle: the resolve step prepends
  'Issue author: @handle' to the AI input and prompts replace
  {issue_author} with it, so comments no longer emit a literal
  placeholder. Re-triage detection switched from 'starts with' to
  'contains' accordingly.
- Remove literal backticks around Next Steps and footer lines in all
  prompts so they render as markdown instead of inline code.
- Close post-processing gaps: 'Valuable Enhancement' surfaces via
  'ai:needs team attention' (no @mention), 'Out of Scope' auto-closes
  as not_planned, and the dead 'needs maintainer input' keyword is
  removed. Summary table updated to match.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@ayeshurun ayeshurun requested a review from a team as a code owner June 21, 2026 12:51
Copilot AI review requested due to automatic review settings June 21, 2026 12:51

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves the GitHub-side AI issue triage infrastructure by fixing prompt routing gaps, strengthening post-processing actions (labels/closing), and making prompts reliably address the issue author. All changes are scoped to .github/ (workflow + prompt YAMLs).

Changes:

  • Added a documentation triage prompt and mapped the documentation label to it, preventing documentation issues from getting stuck in needs triage.
  • Fixed the “author feedback” loop by having AI triage apply the needs author feedback label so policy automation can re-triage on author replies.
  • Updated prompts and workflow input shaping to reliably provide/substitute the issue author handle and improved markdown rendering by removing inline-code formatting around key footer/Next Steps lines.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file
File Description
.github/workflows/ai-issue-triage.yml Prepends Issue author: @… to AI input, adds documentation prompt routing, and improves post-processing (needs-author-feedback labeling, valuable-enhancement surfacing, out-of-scope close reason).
.github/prompts/bug-triage.prompt.yml Updates re-triage detection to “contains” and adjusts Next Steps/footer formatting + author handle substitution guidance.
.github/prompts/feature-triage.prompt.yml Updates re-triage detection to “contains” and adjusts Next Steps/footer formatting + author handle substitution guidance.
.github/prompts/question-triage.prompt.yml Updates re-triage detection to “contains” and adjusts Next Steps/footer formatting + author handle substitution guidance.
.github/prompts/documentation-triage.prompt.yml Introduces a dedicated documentation triage prompt with aligned categories/Next Steps/footer and author handle substitution guidance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants