Skip to content

chore(sep-2640): repoint the requirements at the Skills beta page - #511

Merged
poita merged 4 commits into
modelcontextprotocol:mainfrom
panyam:chore/sep-2640-final-refresh
Nov 10, 2026
Merged

poita merged 4 commits into
modelcontextprotocol:mainfrom
panyam:chore/sep-2640-final-refresh

Conversation

@panyam

@panyam panyam commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

What changes

Repoints src/seps/sep-2640.yaml at the Skills extension's beta page, specification/beta/skills.mdx at 649f7aee. Every row now quotes that page instead of the SEP.

SEP-3392 classified Skills as Beta, and ext-skills#161 moved the page from specification/stable/ to specification/beta/. The WG maintains that page. The SEP is frozen at Final and no longer tracks it. The old stable URL returns 404, and one row pointed there.

No check ID changes, so no scenario logic changes. The three skills scenarios now point their spec reference at the beta page.

This PR started as a re-extraction against the Final SEP. That pass is still in the history (eb5735c, 4b6ff1a), and the beta pass (7065737) replaces its quotes.

Prerequisite knowledge

  • AGENTS.md § Traceability manifest: what a check: row is for, and why traceability.json is generated rather than edited.
  • The Skills beta page. It restates the SEP as a spec page, with new Error Handling and Reservations sections.
  • src/seps/sep-2640.yaml's own header. Its "beta pass" section lists what moved.

Reviewer's guide

Each row quotes a sentence and hangs a check ID on it, and scenarios emit the check IDs. The beta page splits many of the SEP's compound sentences into bullets, so 59 of the 100 rows quoted text the page no longer contains. Most of the diff re-quotes them. A few changes carry meaning.

Read in this order:

  1. src/seps/sep-2640.yaml, the header's "beta pass" section. It lists every substantive change.
  2. The three new checks: sep-2640-directory-read-cache-attributes under Reading Directories, then sep-2640-error-standard-codes and sep-2640-error-informative-messages under Error Handling.
  3. The four removed rows: host-registry-no-fetch, host-virtual-mount-lazy, host-surface-directory-read and sdk-convenience-wrappers. They quoted the SEP's Implementation Guidelines, and the beta page has no such section.
  4. sep-2640-host-verify-digest now verifies size as well as digest. sep-2640-capability-declaration-inline now names server/discover.
  5. src/scenarios/server/skills/helpers.ts, src/scenarios/client/skills/verification.ts and no-prefetch.ts: URL changes only.
  6. The rest is re-quoting. Each row also gained a url: pointing at its section of the beta page.

Decision log

  • Kept the file name, sep: 2640 and the sep-2640- check prefix. Renaming would move every check ID and every baseline that names one. sep-1932.yaml already points spec_url at an ext-auth spec file, so a SEP-numbered yaml quoting an extension page has precedent.
  • Removed the four Implementation Guidelines rows rather than excluding them. Their text is not on the page, so no row can quote it. No scenario emitted any of them. The page's Lazy Retrieval rule (sep-2640-host-no-prefetch) covers what host-registry-no-fetch and host-virtual-mount-lazy required.
  • Dropped the second source. sep-2640-skills-get-cache-attributes used to quote the stable page because the SEP left skills/get caching open. The beta page is now the only source.
  • Did not declare resultType or request _meta. The page says resultType MUST be "complete" on each of its three results, and every request MUST carry request metadata. Both are base-protocol rules restated for the page's own types, and declaring them here would double-count them. The header records this.
  • Added one excluded row, "Hosts SHOULD surface collisions to the user". The SEP folded it into a compound sentence, and the page states it on its own. A harness cannot observe it.
  • Did not touch src/seps/traceability.json. The traceability workflow regenerates it against the reference SDK.

Risk / blast radius

  • Affects: requirement rows, provenance, and three spec-reference URLs. No scenario logic, no runner, no check ID.
  • Tests covering this: npm run typecheck, prettier, and the skills and traceability vitest files (36 tests). The three server scenarios run against a second implementation (mcpkit examples/skills) at 45 pass / 0 fail / 0 warn / 0 skip, unchanged.
  • Quote check: a script matched every row's text: against skills.mdx at 649f7aee, with markdown emphasis and link syntax stripped. All 100 match, except the two Limits rows, which render rows of the page's table.
  • Be paranoid about: the four removed rows, and whether the two Error Handling rows should be one row.

Before / after

Rows quoting text absent from the beta page:

before:  59 of 100
after:    0 of 100 (the two Limits table rows aside)

Rows and coverage, from a traceability run plus the client scenarios:

before after
declared checks 90 89
excluded rows 10 11
emitted by server scenarios 42 42
emitted by client scenarios 4 4

Out of scope

  • Scenarios for the 43 checks no scenario emits. Most are host obligations.
  • Renaming the yaml or its check IDs away from the SEP number.

Refs #513.

The header said to expect one more pass at the CM-stamped head. This is it.
SEP-2640 was marked Accepted on 2026-09-03 (d6b31a03) and Final on 2026-09-11
(8075db4a), and the source moved off the sep/skills-extension branch to main.

Four commits touched the text after this file's extraction point, one of them
normative. The Skill Format section gained a paragraph binding clients to the
Agent Skills specification's own backward-compatibility mechanisms; it is
declared here as two excluded rows, since both halves condition on a breaking
revision of an external spec that does not yet exist, so there is nothing for
a harness to present and nothing for a client to honor.

The Capability Declaration section was reworded to locate the declaration in
the `extensions` field of the server's capabilities rather than loosely in the
`initialize` response, and now states the `resources`-capability requirement
outright instead of leaving it to be derived from Dependencies. Both rows are
restated from the Final text; neither check ID moves, so no scenario changes.

A copy-editing pass (582d814a) swapped em-dashes for commas, colons and
parentheses throughout, which left twelve rows quoting sentences the SEP no
longer contains. Those quotes are re-synced. No requirement changed meaning.

The header's own counts were wrong before this pass and are recomputed: 90
checks and 10 excluded rows, of which the three server scenarios emit 42,
confirmed against a traceability run rather than asserted.

Suite unchanged at 45 pass / 0 fail / 0 warn / 0 skip.
@pkg-pr-new

pkg-pr-new Bot commented Sep 21, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

npx https://pkg.pr.new/@modelcontextprotocol/conformance@511

commit: 7065737

@dfedoryshchev

Copy link
Copy Markdown

Checked the yaml against seps/2640-skills-extension.md at f56f204f.

Nine rows still quote wording that is not in the Final text.

For example, host-nested-fresh-consent quotes:

Activating a nested skill - loading it as a skill in its own right - requires...

while the Final reads:

Activating a nested skill (loading it as a skill in its own right, whether through the host's skill-loading machinery or by giving effect to its frontmatter) requires...

The others:

  • nested-publication-flat
  • skills-list-cache-attributes
  • entry-frontmatter-required
  • host-skill-identity-pair
  • host-content-bound-approval
  • intermediary-meta-prefix
  • sdk-convenience-wrappers
  • directory-read-not-manifest-extension

The drift check from #513 only flags rows that were exact quotes of the old revision, and these were already trimmed there, so it skips them.

skills-get-cache-attributes quotes a source the header does not name.

It quotes "ttlMs and cacheScope are REQUIRED" from the ext-skills stable page (modelcontextprotocol/ext-skills#139). The Final SEP says this "is left open", and the header names the SEP as the only source.

Minor: the resources-capability MUST is new text in fd72d3f1, not derived from Dependencies, so the header undercounts the normative changes.

…nst the Final text

Addresses the review on modelcontextprotocol#511 (2026-10-08).

- Nine rows trimmed in the first pass still carried pre-copy-edit wording
  (em-dash asides the Final text turned into commas, colons and
  parentheses). modelcontextprotocol#513's drift check only catches whole-sentence quotes, so
  it could not see them. All nine now quote f56f204f verbatim.
- A sentence-by-sentence check against f56f204f found four more rows that
  paraphrased rather than quoted: host-origin-tag-visible,
  host-cache-write-isolation, host-cache-excluded-from-discovery and
  host-ignore-allowed-tools. The last had also dropped the "unless the user
  has explicitly approved that grant" exception, which changed its meaning.
- capability-declaration-inline and host-frontmatter-comparison now quote
  the Final sentence instead of a gloss on it.
- skills-get-cache-attributes quotes the ext-skills stable page, because
  the SEP leaves that question open. The header now names that second
  source, and the row's url points at the stable page.
- The header now counts fd72d3f1 as normative: it added the
  resources-capability MUST as new text, not as a restatement of
  Dependencies.

mcpkit examples/skills: 45 pass / 0 fail on this branch.
@panyam

panyam commented Oct 15, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @dfedoryshchev, good catch.
You're right that #513's check couldn't see these, it only flags rows that were whole-sentence quotes of the old revision and these had been trimmed.
I re-quoted all nine against f56f204f in 4b6ff1a.
I also did a sentence-by-sentence pass and found four more that paraphrased rather than quoted: host-origin-tag-visible, host-cache-write-isolation, host-cache-excluded-from-discovery and host-ignore-allowed-tools.
host-ignore-allowed-tools had also lost the "unless the user has explicitly approved that grant" exception, so it was stricter than the SEP.
For skills-get-cache-attributes I kept the row, but the header now names the ext-skills stable page as a second source and the row's url points there, since the SEP leaves it open and ext-skills#139 closed it on stable.
I wonder if longer term this file should extract from skills.mdx instead, since that's what the WG maintains now and ext-skills#157 is about to move it again.
Also counted fd72d3f1 as normative in the header, you're right it's new text and not derived from Dependencies.
mcpkit is still 45/0 on the branch.

@poita

poita commented Nov 10, 2026

Copy link
Copy Markdown

@panyam are you planning on repointing this at the beta text rather than SEP before merging? Should we wait for that before review, or did you want a review before that change?

@panyam

panyam commented Nov 10, 2026

Copy link
Copy Markdown
Contributor Author

@panyam are you planning on repointing this at the beta text rather than SEP before merging? Should we wait for that before review, or did you want a review before that change?

@poita Yep absolutely. I was waiting on #559 to go in first and then rebase this onto that. But yes happening either weay.

SEP-3392 classified Skills as Beta, and ext-skills#161 moved the page to
specification/beta/. The WG maintains that page now, and the old stable
URL returns 404. Every row quotes skills.mdx at 649f7aee.

Most rows are re-quotes. The page splits the SEP's compound sentences into
bullets. Check IDs do not change, so no scenario logic changes. The three
scenario spec references now point at the beta page.

The page has no Implementation Guidelines section, so four rows that
quoted it are removed. No scenario emitted any of them. Three rows are
new: cache attributes on the directory read result, and two from Error
Handling. One excluded row is new, for surfacing name collisions.
@panyam panyam changed the title chore(sep-2640): re-extract against the Final text chore(sep-2640): repoint the requirements at the Skills beta page Nov 10, 2026
@panyam

panyam commented Nov 10, 2026

Copy link
Copy Markdown
Contributor Author

@poita Yep - repointed at the beta page now. Most of the diff is just requoting, the page splits a lot of the SEP's compound sentences into bullets and 59 of the 100 rows no longer were matching. Ready for review now.

@poita

poita commented Nov 10, 2026

Copy link
Copy Markdown

Looks good!

@poita
poita merged commit 96a6f94 into modelcontextprotocol:main Nov 10, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants