Skip to content

Enhance prompts, resources, and tool descriptions based on real-world debugging session#294

Merged
data-douser merged 4 commits into
mainfrom
felickz/enhance-mcp-guidance
Jun 17, 2026
Merged

Enhance prompts, resources, and tool descriptions based on real-world debugging session#294
data-douser merged 4 commits into
mainfrom
felickz/enhance-mcp-guidance

Conversation

@felickz

@felickz felickz commented Jun 15, 2026

Copy link
Copy Markdown
Contributor
  • data-extension-development prompt: add troubleshooting table (compilation cache, CLI discovery, subtypes on stub types, missing chain hops, column count) and Phase 8 note about when to escalate to QL library changes
  • csharp_library_modeling resource: add subtypes flag section with caveat about library stub types (matches existing Java modeling resource pattern)
  • Tool descriptions: clarify register_database (session-scoped validation), codeql_bqrs_decode (array parameter), quick_evaluate (isolation scope)
  • ql-tdd-basic prompt: add compilation cache pitfall

from using MCP on this PR : github/codeql#21984

… debugging session

- data-extension-development prompt: add troubleshooting table (compilation
  cache, CLI discovery, subtypes on stub types, missing chain hops, column
  count) and Phase 8 note about when to escalate to QL library changes
- csharp_library_modeling resource: add subtypes flag section with caveat
  about library stub types (matches existing Java modeling resource pattern)
- Tool descriptions: clarify register_database (session-scoped validation),
  codeql_bqrs_decode (array parameter), quick_evaluate (isolation scope)
- ql-tdd-basic prompt: add compilation cache pitfall

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 15, 2026 15:49
@felickz felickz requested review from a team, data-douser and enyil as code owners June 15, 2026 15:49
@github-actions

github-actions Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates MCP prompt/reference content and tool descriptions based on practical debugging learnings, aiming to make CodeQL modeling workflows and tool usage clearer for users of the CodeQL Development MCP Server.

Changes:

  • Expanded prompt guidance for data-extension development and TDD, including troubleshooting and compilation-cache pitfalls.
  • Added C# library modeling guidance for the subtypes flag (with a caveat about stub/framework base types).
  • Refined several CodeQL tool parameter/behavior descriptions to reduce user confusion.
Show a summary per file
File Description
server/src/tools/codeql/register-database.ts Updates register_database tool description to better communicate validation/usage scope.
server/src/tools/codeql/quick-evaluate.ts Updates quick_evaluate tool description to clarify isolation/usage intent.
server/src/tools/codeql/bqrs-decode.ts Clarifies that the files parameter is an array (even for a single BQRS).
server/src/resources/languages/csharp_library_modeling.md Adds guidance and caveats for the subtypes flag in C# MaD tuples.
server/src/prompts/ql-tdd-basic.prompt.md Adds a common pitfall about compilation cache masking data extension changes.
server/src/prompts/data-extension-development.prompt.md Adds escalation guidance (“when data extensions aren’t enough”) and a troubleshooting table.

Copilot's findings

  • Files reviewed: 6/6 changed files
  • Comments generated: 3

Comment thread server/src/tools/codeql/register-database.ts Outdated
Comment thread server/src/tools/codeql/quick-evaluate.ts Outdated
Comment thread server/src/tools/codeql/bqrs-decode.ts Outdated
- register_database: remove 'in this session' claim — tool only validates,
  does not persist session state. Clarify it's a preflight check.
- quick_evaluate: remove claim that it evaluates against the database —
  current implementation only resolves symbol position. Note that
  codeql_query_run should be used for full evaluation.
- codeql_bqrs_decode: fix 'path(s)' to 'paths' per grammar feedback.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@felickz felickz enabled auto-merge (squash) June 15, 2026 22:28
@data-douser

Copy link
Copy Markdown
Collaborator

@copilot Tidy up this PR and rebuild server/dist/** in order to fix failing lint, format, and build checks for this PR. The full npm run build:all script should fix all of this in one rather long run, or individually run npm run tidy and npm run build to fix lint/format and build errors, respectively.

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
auto-merge was automatically disabled June 16, 2026 22:16

Head branch was pushed to by a user without write access

Copilot AI review requested due to automatic review settings June 16, 2026 22:16
@felickz felickz removed the request for review from Copilot June 16, 2026 22:16

Copilot AI commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

@copilot Tidy up this PR and rebuild server/dist/** in order to fix failing lint, format, and build checks for this PR. The full npm run build:all script should fix all of this in ...

Fixed in 1db4e12. Prettier reformatted the troubleshooting table in data-extension-development.prompt.md (the only format failure), and server/dist/** was rebuilt. npm run lint -w server, npm run format:check, and npm run lint:md all pass now.

@data-douser

Copy link
Copy Markdown
Collaborator

@copilot Fix the tests for this repo so that all tests are passing when npm run build:all is run from repo root. There is at least one test currently failing:

⎯⎯⎯⎯⎯⎯⎯ Failed Tests 1 ⎯⎯⎯⎯⎯⎯⎯

 FAIL  test/src/tools/codeql/quick-evaluate.test.ts > registerQuickEvaluateTool > should register the quick_evaluate tool with the MCP server
AssertionError: expected "vi.fn()" to be called with arguments: [ 'quick_evaluate', …(3) ]

Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 16, 2026 22:41
@felickz felickz removed the request for review from Copilot June 16, 2026 22:41

Copilot AI commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

@copilot Fix the tests for this repo so that all tests are passing when npm run build:all is run from repo root. There is at least one test currently failing:

...

Fixed in 520a6e5. The quick_evaluate registration test still asserted the old description string; updated it to match the revised description in quick-evaluate.ts. All 1486 server tests pass now.

@data-douser data-douser left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the improvements to ql-mcp @felickz -- LGTM!

@data-douser data-douser merged commit 5e4df14 into main Jun 17, 2026
19 of 20 checks passed
@data-douser data-douser deleted the felickz/enhance-mcp-guidance branch June 17, 2026 16:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants