Skip to content

FEAT: Expose selected PyRIT REST operations over MCP - #4419

Open
varunj-msft wants to merge 2 commits into
microsoft:mainfrom
varunj-msft:13538-mcp-endpoint
Open

varunj-msft wants to merge 2 commits into
microsoft:mainfrom
varunj-msft:13538-mcp-endpoint

Conversation

@varunj-msft

@varunj-msft varunj-msft commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Description

1 of 2. The skill PR, #4420, builds on this one.

Design: Status after the design discussion (try both an MCP endpoint and skills), Rules for the endpoint.

Lets an AI agent such as GitHub Copilot use PyRIT through the backend: a new MCP endpoint at /api/mcp serves a selected set of REST operations as tools.

Problem. An agent has no supported way to use PyRIT except by driving the CLI or the GUI. The design discussion settled on trying both an MCP endpoint and skills; this PR is the endpoint.

Change.

  • RestToolset (pyrit/backend/rest_tools.py) offers 14 REST operations as tools: listing targets, scenarios, techniques, datasets, and scorers; starting and following scenario runs; and reading attacks, conversations, and analytics. Each tool takes its name, description, and input schema from the OpenAPI document. A call is an in-process REST request with the caller's Authorization header, so it gets the same validation, errors, authentication, and runtime admission as any other request. A path parameter can't contain /, so a call can't reach a route that isn't offered.
  • MCPEndpoint (pyrit/backend/mcp_server.py) serves the tools over Streamable HTTP, with a short PyRIT guide as the server instructions. It answers with JSON and accepts only POST: a GET would open an event stream that stays connected, and the backend counts it as active work, which blocks live configuration apply. A browser request must come from an origin in PYRIT_CORS_ORIGINS, checked the way CORS checks it, as the MCP spec requires; CORS now also allows the MCP-Protocol-Version header those clients send.
  • Responses are shortened for the model:
    • Identifiers keep the hash that a request needs to pin a registered component, such as objective_scorer.
    • Null values and empty lists and objects are dropped. Empty strings stay, because some are real values, such as an analytics bucket for an empty label.
    • Text over 1,000 characters is a preview unless its field is named in response_fields. If the full text doesn't fit, previews come back instead.
    • A response over 40,000 characters keeps the leading entries of its largest list and drops continuation cursors that would skip the rest.
    • A response_fields path that the response can't contain is reported with the fields that are available.
  • Docs: pyrit/backend/README.md and a short "Using an AI Agent" section in the scanner docs.

What stays the same. No REST route changes. Every route except /api/mcp still requires the compatibility header (MCP clients aren't PyRIT builds). Authentication, admission, and request limits apply as for any /api request.

This starts from Behnam's prototype (behnam-o/PyRIT@8f500ac), without its Agent page, which is separate work.

Demonstration (local backend, MCP Python SDK client):

POST /api/mcp, Origin http://attacker.example        -> 403
GET  /api/mcp                                        -> 405, Allow: POST
list_scorers(response_fields=[items.scorer_registry_name, items.identifier.hash])  -> 28 scorers
start_scenario_run(technique.execute, techniques=[flip],
    objective_scorer={registry_name: task_achieved_refined_gpt4o_temp9, identifier_hash: ...})
get_scenario_run                                     -> COMPLETED, 1 of 1 attacks succeeded
get_conversation_messages(response_fields=[messages.message_pieces.original_value])
                                                     -> the 1,524-character reply in full
get_conversation_messages for many_shot (prompt ~140,000 characters), same fields
                                                     -> prompt previewed, reply in full

With GitHub Copilot CLI: copilot mcp add --transport http pyrit http://localhost:8000/api/mcp.

Serve a curated set of backend REST operations as MCP tools at /api/mcp,
so AI agents such as GitHub Copilot can list targets, scenarios, techniques,
datasets, and scorers, start and follow scenario runs, and read attack
results, conversations, and analytics.

- Generate each tool from the OpenAPI document and call it in-process with
  the caller's credentials, so it gets the same validation, errors,
  authentication, and runtime admission as any REST request.
- Accept only POST: responses are JSON, and a GET event stream would stay
  open and count as active work.
- Check browser origins against PYRIT_CORS_ORIGINS as CORS does, and let
  browser clients send the MCP protocol version header.
- Keep component identifier hashes, which pinned references such as
  objective_scorer need, and drop eval_hash.
- Keep empty strings, which can be real values, and drop other empty values.
- Return named response_fields in full, falling back to previews when they
  do not fit, and report a named field as unknown only when the response
  cannot contain it.
- Keep oversized responses valid JSON by cutting the largest list and
  dropping continuation fields that would skip the cut entries.
- Reject path parameters that contain "/", which would reach other routes.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A critical route-collision flaw allows tool calls to reach unoffered operations, bypassing the curated operation list and returning unexpected response shapes.

1 open issue
What changed in this PR

Exposes selected PyRIT backend REST operations through /api/mcp, letting AI agents discover components, run scenarios, and inspect results. This provides the endpoint used by the follow-up skill in #4420.

Changes:

  • Generates 14 tools from OpenAPI and dispatches calls in-process with caller credentials.
  • Adds a POST-only MCP endpoint with origin checks and compact, field-selectable responses.
  • Documents client setup and adds tests for tool behavior, transport, and authentication.
File Description
tests/​unit/​backend/​test_rest_tools.py Tests tool schemas, dispatch, response compaction, and component hashes.
tests/​unit/​backend/​test_mcp_server.py Tests MCP calls, lifespan, origin checks, and transport restrictions.
tests/​unit/​backend/​test_compatibility.py Checks MCP authentication without a compatibility marker.
pyrit/​backend/​rest_tools.py Generates selected REST tools and formats their responses.
pyrit/​backend/​README.md Documents tools, MCP setup, and compatibility behavior.
pyrit/​backend/​middleware/​compatibility.py Exempts MCP from compatibility-marker enforcement.
pyrit/​backend/​mcp_server.py Implements the MCP transport and tool handlers.
pyrit/​backend/​main.py Registers MCP, manages its lifespan, and updates CORS headers.
doc/​scanner/​0_scanner.md Adds AI-agent connection and authentication guidance.

Review effort: Balanced

Comment on lines +145 to +148
nested = [parameter for parameter in operation.path_parameters if "/" in str(arguments[parameter])]
if nested:
return json.dumps({"error": f"Path parameters cannot contain '/': {', '.join(nested)}."}), True
response = await self._send_async(operation=operation, arguments=arguments, authorization=authorization)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants