Consume BCQuality knowledge base for Copilot PR review#8700
Draft
WaelAbuSeada wants to merge 5 commits into
Draft
Consume BCQuality knowledge base for Copilot PR review#8700WaelAbuSeada wants to merge 5 commits into
WaelAbuSeada wants to merge 5 commits into
Conversation
Replace the local al-code-review skill and instructions with the BCQuality consumption model: the workflow now checks out microsoft/BCQuality at main, and the review script copies that clone into the runner workspace and instructs Copilot CLI to start from BCQuality skills/entry.md. The script's flat JSON output contract is preserved so the comment-posting pipeline is unchanged.
Replace the naive direct-checkout bridge with the config-driven clone+filter pattern from the tested reference implementation: - Add tools/BCQuality integration layer (config YAML, Get-BCQualityConfig, Invoke-BCQualityFilter, README) - Replace orchestrator with the version that consumes BCQuality native structured DO output (agent findings, references, confidence, interrupted-JSON repair, filter report in PR summary) - Runner workflow now clones the configured BCQuality repo/ref, filters it to policy, and passes BCQUALITY_ROOT/SHA + BCQUALITY_* + AGENT_MINIMUM_SEVERITY; uploads the filter report
github-actions
Bot
added
the
Build: Automation
JesperSchulz
approved these changes
Jun 20, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
This PR migrates the Copilot PR review orchestration to consume centrally maintained review skills/knowledge from microsoft/BCQuality at runtime, replacing the duplicated in-repo instruction/skill content with a config-driven clone + filter layer.
Changes:
- Added
tools/BCQuality/integration layer (config + config loader + deterministic filter with audit report artifact). - Updated the PR review runner workflow to fetch/filter BCQuality and run the orchestrator with BCQuality as the working directory (pinned ref for reproducibility).
- Removed the legacy in-repo AL review skill and domain instruction markdown files now sourced from BCQuality.
Reviewed changes
Copilot reviewed 13 out of 14 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
.github/workflows/CopilotPRReviewRunner.yaml |
Adds BCQuality fetch/filter, copilot-requests: write, and switches Copilot CLI auth to built-in token. |
tools/BCQuality/bcquality.config.yaml |
Pins BCQuality ref and defines enabled layers + knowledge allow/deny + task-context defaults. |
tools/BCQuality/scripts/Get-BCQualityConfig.ps1 |
Loads YAML config and applies env overrides for Actions variables. |
tools/BCQuality/scripts/Invoke-BCQualityFilter.ps1 |
Prunes cloned BCQuality content and emits _filter-report.json. |
tools/BCQuality/README.md |
Documents the shared BCQuality integration layer and override schema. |
tools/Code Review/scripts/Invoke-CopilotPRReview.ps1 |
Orchestrator now boots via BCQuality skills/entry.md and parses BCQuality DO-contract output. |
tools/Code Review/README.md |
Documents the updated two-workflow pattern, severity mapping, and BCQuality-backed findings model. |
tools/Code Review/skills/al-code-review/SKILL.md |
Removed (now sourced from BCQuality). |
tools/Code Review/instructions/accessibility.md |
Removed (now sourced from BCQuality). |
tools/Code Review/instructions/performance.md |
Removed (now sourced from BCQuality). |
tools/Code Review/instructions/privacy.md |
Removed (now sourced from BCQuality). |
tools/Code Review/instructions/security.md |
Removed (now sourced from BCQuality). |
tools/Code Review/instructions/style.md |
Removed (now sourced from BCQuality). |
tools/Code Review/instructions/upgrade.md |
Removed (now sourced from BCQuality). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+92
to
+94
| elseif ('.\+(){}[]^$|' -contains $c) { | ||
| $null = $regex.Append('\').Append($c) | ||
| } |
Comment on lines
+21
to
+26
| The review uses the `pull_request` → `workflow_run` privilege-escalation | ||
| pattern (as in the original BCApps port). The unprivileged intake job | ||
| fires on `pull_request` events from untrusted forks; the runner job fires | ||
| on `workflow_run` events, runs against the **trusted base branch**, and | ||
| only then exposes `COPILOT_GH_TOKEN`. This eliminates the | ||
| `pull_request_target` attack surface entirely. |
Comment on lines
+130
to
+135
| ## Required secrets | ||
|
|
||
| | Secret | Purpose | | ||
| |---|---| | ||
| | `COPILOT_GH_TOKEN` | Copilot-enabled PAT scoped to the `copilot-pr-review` environment. Used **only** by the Copilot CLI subprocess for authentication. | | ||
|
|
Comment on lines
149
to
152
| env: | ||
| GITHUB_TOKEN: ${{ github.token }} | ||
| GH_TOKEN: ${{ secrets.COPILOT_GH_TOKEN }} | ||
| GH_TOKEN: ${{ github.token }} | ||
| GITHUB_REPOSITORY: ${{ github.repository }} |
darjoo
reviewed
Jun 21, 2026
Comment on lines
+96
to
+98
| if (-not $cfg.ContainsKey('bcquality') -or -not ($cfg['bcquality'] -is [hashtable])) { $cfg['bcquality'] = @{} } | ||
| if (-not $cfg['bcquality'].ContainsKey('repo')) { $cfg['bcquality']['repo'] = 'https://github.com/microsoft/BCQuality' } | ||
| if (-not $cfg['bcquality'].ContainsKey('ref')) { $cfg['bcquality']['ref'] = 'main' } |
Contributor
There was a problem hiding this comment.
These are very hard to read as they're all inline.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Migrates the Copilot PR reviewer from a self-maintained instructions/skill model to consuming the proven, centrally-maintained BCQuality knowledge base through a small, config-driven clone + filter integration layer.
The reviewer now pulls its review knowledge (performance, privacy, security, style, upgrade) from
microsoft/BCQualityat runtime, instead of carrying a duplicated copy of that guidance in this repo.What changed
Added — BCQuality integration layer (
tools/BCQuality/)bcquality.config.yaml— single source of truth for which BCQuality content this repo consumes (repo/ref, enabled layers, disabled skills, knowledge allow/deny, task-context). Every value is overridable at runtime via Actions variables.scripts/Get-BCQualityConfig.ps1— loads the YAML and appliesBCQUALITY_*env overrides.scripts/Invoke-BCQualityFilter.ps1— prunes the BCQuality clone to policy and emits a_filter-report.jsonartifact for auditability.README.md— documents the layer.Updated — orchestrator (
tools/Code Review/scripts/Invoke-CopilotPRReview.ps1)BCQUALITY_ROOT/BCQUALITY_SHA) and reads BCQuality'sentry.mdrather than the in-repo instruction files.Updated — runner workflow (
.github/workflows/CopilotPRReviewRunner.yaml)GITHUB_TOKEN+copilot-requests: writeto bill Copilot inference to the org (no PAT secret required).init + fetch + checkoutso the pinned ref may be a branch, tag, or commit SHA.Config
refis pinned to a specificmaincommit (822cae1b2771ac25f665f73369f69093bd4fd630) for reproducible reviews; bump deliberately as BCQuality advances.Removed — now sourced from BCQuality
tools/Code Review/instructions/*.md(accessibility, performance, privacy, security, style, upgrade)tools/Code Review/skills/al-code-review/SKILL.mdFixes AB#637778