Skip to content

Add list_code_scanning_alerts query-bounding guardrail and regression tests#46198

Draft
pelikhan with Copilot wants to merge 5 commits into
mainfrom
copilot/hippo-add-regression-coverage-again
Draft

Add list_code_scanning_alerts query-bounding guardrail and regression tests#46198
pelikhan with Copilot wants to merge 5 commits into
mainfrom
copilot/hippo-add-regression-coverage-again

Conversation

Copilot AI commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Unbounded list_code_scanning_alerts MCP calls return oversized payloads that break downstream workflow runs. Both GitHub MCP tools prompt files now instruct agents to always include state: open and severity: critical,high.

Changes

  • actions/setup/md/github_mcp_tools_prompt.md — added **Code scanning queries** guardrail line requiring state: open and severity: critical,high on every list_code_scanning_alerts call
  • actions/setup/md/github_mcp_tools_with_safeoutputs_prompt.md — same guardrail in the safe-outputs variant
  • pkg/workflow/prompt_validation_test.go — three new regression tests:
    • TestGitHubMCPToolsPromptBoundsCodeScanningAlertQueries — asserts the plain prompt file contains both required parameters
    • TestGitHubMCPToolsWithSafeOutputsPromptBoundsCodeScanningAlertQueries — same assertion on the with-safeoutputs variant
    • TestGitHubMCPToolsPromptIncludedForCodeSecurityToolset — compiles a code_security-toolset workflow and verifies the generated lock file references one of the guardrail-carrying prompt files

The guardrail text added to both prompt files:

**Code scanning queries**: When calling `list_code_scanning_alerts`, always include
`state: open` and `severity: critical,high` to bound the response size and avoid
oversized payloads.

Copilot AI and others added 2 commits July 17, 2026 10:01
…guardrails

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
…rails

Add state: open and severity: critical,high guardrail to both GitHub MCP
tools prompt files to prevent unbounded code-scanning queries that produce
oversized responses and break downstream workflow runs.

Add three regression tests in prompt_validation_test.go:
- TestGitHubMCPToolsPromptBoundsCodeScanningAlertQueries
- TestGitHubMCPToolsWithSafeOutputsPromptBoundsCodeScanningAlertQueries
- TestGitHubMCPToolsPromptIncludedForCodeSecurityToolset

Closes #46162

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Add regression tests for code scanning alert queries Add list_code_scanning_alerts query-bounding guardrail and regression tests Jul 17, 2026
Copilot AI requested a review from pelikhan July 17, 2026 10:19
@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot use the field selection feature from the GitHub MCP server 1.6.0 to apply these APIs. Update prompt.

Copilot AI and others added 2 commits July 17, 2026 11:34
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
…er 1.6.0)

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Jul 17, 2026

Copy link
Copy Markdown
Contributor Author

@copilot use the field selection feature from the GitHub MCP server 1.6.0 to apply these APIs. Update prompt.

Done in commit feat: add fields_param guidance to GitHub MCP tools prompts (MCP server 1.6.0).

Both prompt files now include a Field selection guardrail documenting the fields parameter introduced in GitHub MCP server 1.6.0 (available via the fields_param feature flag, auto-enabled in Insiders mode):

**Field selection (GitHub MCP server ≥ 1.6.0)**: When calling list/search tools that accept a `fields` parameter
(`list_pull_requests`, `list_issues`, `search_issues`, `search_pull_requests`, `list_commits`, `list_releases`,
`search_code`, `get_file_contents`), pass only the fields you need to reduce response size.
Example: `list_pull_requests` with `fields: [number, title, state, html_url]`.

Note: list_code_scanning_alerts does not have a fields parameter in v1.6.0 (not in the fields_param insiders feature set), so that tool continues to use the state: open / severity: critical,high filtering guardrail.

The SKILL.md Pull Requests section is also updated to prefer the fields approach over minimal_output for servers ≥ 1.6.0. A new regression test TestGitHubMCPToolsPromptHasFieldSelectionGuidance verifies the guidance is present in both prompt files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[hippo] Add regression coverage so generated code-scanning prompts always bound alert queries

2 participants