Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .github/workflows/conformance.yml
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,45 @@ jobs:
--spec-version 2025-11-25 \
--expected-failures ./conformance-tests/conformance-baseline.yml

modern-client:
name: Modern Client Conformance (2026-07-28)
runs-on: ubuntu-latest
strategy:
matrix:
scenario:
- tools_call
- request-metadata
- sep-2322-client-request-state
- http-standard-headers
- http-custom-headers
- http-invalid-tool-headers
- json-schema-ref-no-deref
- json-schema-2020-12-preservation
steps:
- uses: actions/checkout@v7

- name: Set up JDK 17
uses: actions/setup-java@v6
with:
java-version: '17'
distribution: 'temurin'
cache: 'maven'

- name: Build client
run: ./mvnw clean install -DskipTests

- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '22'

- name: Run conformance test
run: |
npx --yes "$CONFORMANCE" client \
--command 'java -jar conformance-tests/client-modern/target/client-modern-*-SNAPSHOT.jar' \
--scenario ${{ matrix.scenario }} \
--spec-version 2026-07-28

auth:
name: Auth Conformance
runs-on: ubuntu-latest
Expand Down
26 changes: 26 additions & 0 deletions conformance-tests/VALIDATION_RESULTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Last validated: **2026-10-02** against conformance suite
## Summary

**Modern Server Tests (2026-07-28 requirements):** 37/37 required scenarios passed (`server-servlet-modern`)
**Modern Client Tests (2026-07-28):** 8/8 non-auth client scenarios passed, 62/62 checks (`client-modern`)
**Server Tests (active suite):** 73/73 checks passed (31 scenarios, 100%)
**Server Tests (SEP-1613 `json-schema-2020-12`):** 5/5 checks passed (SEP-2106 checks skipped — post-2025-11-25 spec additions)
**Client Tests:** 3/4 scenarios passed; `sse-retry` fails (tracked in `conformance-baseline.yml`)
Expand Down Expand Up @@ -44,6 +45,31 @@ in the anchor release (`0.2.0-alpha.10`) as `not_scored`. Pending scenarios are
`http-custom-header-server-validation` (10/10, SEP-2243 custom headers, against the `test_x_mcp_header` tool)
- **Failing — tasks extension (`tasks-*`, SEP-2663):** not implemented

## Modern Client Test Results (2026-07-28)

The `client-modern` module drives `io.modelcontextprotocol.modern.client.McpSyncClient` over
`HttpClientMcpTransport`. Last validated on **2026-11-05** with `--spec-version 2026-07-28`.

### Passing (8/8 scenarios, 62/62 checks)

- **tools_call (2/2):** tool discovery and invocation
- **request-metadata (6/6, SEP-2575):** `_meta` with version, capabilities and client info on every
request, `MCP-Protocol-Version` matching `_meta`, retry after `-32022`
- **sep-2322-client-request-state (5/5, SEP-2322):** `requestState` echoed byte for byte and omitted
when absent, a new JSON-RPC id per retry, no MRTR state on other requests, missing `resultType` as
`complete`
- **http-standard-headers (9/9, SEP-2243):** `Mcp-Method` and `Mcp-Name` on tools, resources and prompts
- **http-custom-headers (18/18, SEP-2243):** `Mcp-Param-*` mirroring, Base64 encoding of unsafe values,
omitted `null` arguments
- **http-invalid-tool-headers (12/12, SEP-2243):** tools with invalid `x-mcp-header` annotations left out,
valid tools kept
- **json-schema-ref-no-deref (1/1, SEP-2106)** and **json-schema-2020-12-preservation (9/9, SEP-1613)**

### Not covered

- **auth/\*:** the modern client has no OAuth support of its own; authorization plugs in through
`McpHttpRequestCustomizer` and `McpHttpAuthorizationErrorHandler`.

## Server Test Results

### Active Suite — Passing (31/31 scenarios, 73/73 checks)
Expand Down
41 changes: 41 additions & 0 deletions conformance-tests/client-modern/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# MCP Conformance Tests - Modern Client

Conformance client for the stateless **2026-07-28** MCP revision, built on the synchronous
`io.modelcontextprotocol.modern.client.McpSyncClient` over `HttpClientMcpTransport`. The legacy
(2025-11-25) clients live in [`client-jdk-http-client`](../client-jdk-http-client) and
[`client-spring-http-client`](../client-spring-http-client).

Each scenario uses the client as an application would: list, call, read, get. The protocol details the
suite checks are the client's job, so the scenarios contain none of them.

## Scenarios

| Scenario | What the client does |
|---|---|
| `tools_call` | Lists the tools and calls the first one |
| `request-metadata` | `server/discover`, then `tools/list`; the server rejects the first request with `-32022` |
| `sep-2322-client-request-state` | Continues one call round by round with `callOnce()`, calling another tool in between, and lets the client run the others with its elicitation handler |
| `http-standard-headers` | Lists and uses every tool, resource and prompt |
| `http-custom-headers` | Calls the tools named in `MCP_CONFORMANCE_CONTEXT` with their arguments |
| `http-invalid-tool-headers` | Calls every listed tool; tools with invalid `x-mcp-header` annotations aren't listed |
| `json-schema-ref-no-deref` | Lists the tools |
| `json-schema-2020-12-preservation` | Sends the listed input schema back through `json_schema_echo` |

The `auth/*` scenarios of the 2026-07-28 requirement set aren't covered: the modern client has no OAuth
support of its own, only the HTTP hooks an authorization integration plugs into.

## Running

```bash
./mvnw clean install -DskipTests

npx @modelcontextprotocol/[email protected] client \
--command "java -jar conformance-tests/client-modern/target/client-modern-2.2.0-SNAPSHOT.jar" \
--scenario http-custom-headers \
--spec-version 2026-07-28
```

The client reads the scenario from `MCP_CONFORMANCE_SCENARIO`, optional scenario data from
`MCP_CONFORMANCE_CONTEXT`, and the server URL from its last argument. It logs to stderr.

See [VALIDATION_RESULTS.md](../VALIDATION_RESULTS.md) for the latest results.
89 changes: 89 additions & 0 deletions conformance-tests/client-modern/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>io.modelcontextprotocol.sdk</groupId>
<artifactId>conformance-tests</artifactId>
<version>2.2.0-SNAPSHOT</version>
</parent>
<artifactId>client-modern</artifactId>
<packaging>jar</packaging>
<name>MCP Conformance Tests - Modern Client</name>
<description>Conformance client for the modern (2026-07-28) Java MCP SDK client</description>
<url>https://github.com/modelcontextprotocol/java-sdk</url>

<scm>
<url>https://github.com/modelcontextprotocol/java-sdk</url>
<connection>scm:git:git://github.com/modelcontextprotocol/java-sdk.git</connection>
<developerConnection>scm:git:ssh://[email protected]/modelcontextprotocol/java-sdk.git</developerConnection>
</scm>

<properties>
<module.name>io.modelcontextprotocol.sdk.conformance.client.modern</module.name>
<maven.deploy.skip>true</maven.deploy.skip>
</properties>

<dependencies>
<dependency>
<groupId>io.modelcontextprotocol.sdk</groupId>
<artifactId>mcp-core</artifactId>
<version>2.2.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>io.modelcontextprotocol.sdk</groupId>
<artifactId>mcp-json-jackson3</artifactId>
<version>2.2.0-SNAPSHOT</version>
</dependency>

<!-- Logging -->
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>${logback.version}</version>
<scope>runtime</scope>
</dependency>
</dependencies>

<build>
<plugins>
<!-- Maven Shade Plugin for creating executable JAR with dependencies -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.6.2</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>
io.modelcontextprotocol.conformance.client.modern.ModernConformanceClient</mainClass>
</transformer>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer" />
</transformers>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>

</project>
Loading
Loading