diff --git a/README.md b/README.md index b021b0b..ab32e15 100644 --- a/README.md +++ b/README.md @@ -2,9 +2,7 @@ # patch-it-plugins -**How patch-it notices, on its own, that something your code depends on changed.** Each plugin watches one provider's own source (an API spec, a changelog, a shutdown notice, a release feed) and turns what changed into a change event, with the source linked and quoted. - -**Why:** when anything your code depends on changes (APIs, packages, SDKs, runtimes, CI, AI models), [patch-it](https://patch-it.dev) finds the code that breaks and opens the pull request that fixes it. In your CI, on your key. You get the fix, not the alert. These plugins are the trigger: nobody has to assign an alert or open a ticket. +**How patch-it notices, on its own, that something your code depends on changed.** Each provider (Stripe, OpenAI, GitHub…) has sources (a spec, a changelog, a shutdown notice, a feed); a plugin reads every version of a source and turns what changed into a change event, with the source linked and quoted. [![CI](https://github.com/patch-it-dev/patch-it-plugins/actions/workflows/CI_go.yml/badge.svg)](https://github.com/patch-it-dev/patch-it-plugins/actions/workflows/CI_go.yml) [![Go Reference](https://pkg.go.dev/badge/github.com/patch-it-dev/patch-it-plugins.svg)](https://pkg.go.dev/github.com/patch-it-dev/patch-it-plugins) @@ -13,110 +11,27 @@ [![Docs](https://img.shields.io/badge/docs-docs.patch--it.dev-3f5bf0)](https://docs.patch-it.dev) [![Discussions](https://img.shields.io/github/discussions/patch-it-dev/patch-it-plugins)](https://github.com/patch-it-dev/patch-it-plugins/discussions) -```mermaid -flowchart LR - A["Provider's own source
spec · changelog · shutdown notice · feed"] -->|"read daily"| B["Plugin
(this repo)"] - B -->|"what changed, quoted"| C["Change event
breaking · deprecation · date · source"] - C --> D["patch-it in your CI
finds the lines, an AI model writes the change"] - D --> E["Pull request in your repo
your CI runs your tests, you merge it"] -``` - -## What patch-it does - -1. **It notices on its own.** A plugin here sees the change where the provider announces it, and patch-it picks it up from there: no alert to triage, no ticket to assign. That includes changes that never bump a version, like an AI model or an API shut down on a date. -2. **It writes the fix.** An AI model (on your own key) reads the change and the lines it affects and writes the code change. It arrives as a pull request: your CI runs your tests on it, like any PR, and you review and merge it. The code change, not just the version. -3. **Proof:** 8 pull requests opened by the early version have been merged by outside maintainers. -4. **What it covers:** APIs, AI models, packages and SDKs, runtimes and CI. The current list of providers and sources is generated from the code: see [docs.patch-it.dev](https://docs.patch-it.dev/supported-apis/plugins/), or run `make plugins`. - -5. **Start:** [docs.patch-it.dev](https://docs.patch-it.dev) sets up the GitHub Action in your repo. To have it watch a provider it doesn't cover yet, add a plugin (below). - ## Add a provider in 15 minutes -You need Go (the version in `go.mod`), `make` and Python 3. Pick the format that reads your provider's source ([which one](docs/writing-a-plugin.md#3-pick-the-format-plugin)), then: - -**1. Scaffold it.** - -```sh -URL=https://acme.example/changelog.xml # your provider's source -go run ./cmd/new-plugin --api acme --provider Acme --format feed --source "$URL" -``` -``` -Scaffolded acme (Acme, format feed): - plugins/feed/acme.go (new) - plugins/feed/acme_test.go (new) - ... -Next steps (every place to fill in says TODO(new-plugin): ...) -``` - -**2. Save a real copy of the source and write the golden file.** Fill in the `TODO(new-plugin)`s: the reader in `plugins/feed/acme.go`, the saved file's row in `testdata/README.md`, and "Why this source" in `providers/acme.md`. - -```sh -curl -fsSL -o plugins/feed/testdata/acme.rss.xml "$URL" -go test ./plugins/feed -run TestAcme -update # writes testdata/golden/acme.json: read it like code -``` - -**3. Check it, the way CI will.** - -```sh -make check -``` -``` -... -make check: all passed -``` - -**4. Open the pull request** (`git commit -s`, one plugin per PR). Details: [docs/writing-a-plugin.md](docs/writing-a-plugin.md). Stuck? Ask in [Discussions](https://github.com/patch-it-dev/patch-it-plugins/discussions). Looking for a first change? [Good first issues](https://github.com/patch-it-dev/patch-it-plugins/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22). - -## How a plugin works - -1. **Fetch.** The plugin asks its environment for the source (`env.Get`); it never opens connections itself. Pages that change in place are also cited through a Web Archive snapshot, so the link in an event keeps showing what the plugin read. -2. **Compare.** It compares the source to the last version it saw (the baseline): two OpenAPI specs, two versions of a deprecations page, a new release's changelog section. -3. **Emit events.** Each change becomes one event in [patch-it's change-event format](https://github.com/patch-it-dev/patch-it/tree/main/docs/contract): what changed, how bad it is (breaking, deprecation, behavior change, non-breaking), the date it takes effect, and the source URL with a quote. **No source, no event.** When unsure, a plugin emits nothing: a wrong event would become a wrong pull request. - -Most providers are an entry for a shared format plugin (`openapi`, `retirements`, `migrationguide`, `removedmembers`, `changelog`, `feed`, `apichangelog`, `catalog`), not new code. patch-it's hosted ingestion service runs every plugin daily and serves the events to the patch-it GitHub Action and CLI. - -## Supported plugins - -There's no list in this repo: each provider and source is registered in code with its metadata (`plugin.RegisterProvider`). `make plugins` prints the list as JSON, every release attaches it as `plugins.json`, and [docs.patch-it.dev](https://docs.patch-it.dev/supported-apis/plugins/) shows it. - -## Layout - -| Path | What's there | -|---|---| -| [`core/plugin/`](core/plugin/) | The `Plugin` interface, the registry (`Register`, `Lookup`, `Names`), `Env`, `Result` | -| [`plugins/`](plugins/) | The format plugins; `plugins/all` imports every one | -| [`cmd/new-plugin/`](cmd/new-plugin/), [`cmd/list-plugins/`](cmd/list-plugins/) | The starter for a new provider; the list of providers and sources, printed from the registry | -| [`core/kit/`](core/kit/), [`core/kit/kittest/`](core/kit/kittest/), [`core/github/`](core/github/) | What plugins share (the unchanged-page check, Wayback citing, the newest release or commit on GitHub) and the test kit (in-memory store, saved pages, golden files) | -| [`core/event/`](core/event/), [`core/migrate/`](core/migrate/) | The change-event types of [patch-it's contract](https://github.com/patch-it-dev/patch-it/tree/main/docs/contract), and the v1 → v2 conversion | -| [`core/diff/`](core/diff/), [`core/normalize/`](core/normalize/), [`core/hint/`](core/hint/), [`plugins/openapi/adapter/`](plugins/openapi/adapter/) | The OpenAPI diff engine: compare two specs, turn the changes into events, attach changelog hints; one adapter per spec provider | -| [`plugins/retirements/retire/`](plugins/retirements/retire/), [`plugins/migrationguide/sdkguide/`](plugins/migrationguide/sdkguide/), [`core/fetch/`](core/fetch/) | The deprecations-page and migration-guide readers; the HTTP fetch and cache | -| [`contract/fixtures/`](contract/) | A copy of patch-it's change-event fixtures | -| `plugin/`, `event/`, `fetch/`, `migrate/`, `kit/`, `kit/kittest/`, `retire/`, `sdkguide/` | Deprecated: the old import paths, forwarding to the packages above until code that imports them moves | -| [`providers/`](providers/) | One page per provider: why that source, what the plugin reads | -| [`docs/`](docs/) | [How to write a plugin](docs/writing-a-plugin.md), the [README template](docs/README-template.md) patch-it's public repos share | - -## Run the checks - -```sh -make check # what CI runs: tidy, build, gofmt, vet, import and net/http guards, go test -race, no TODO(new-plugin) left, leak guard -``` - -The tests are offline: every plugin's test reads a saved copy of its source from `testdata/` and compares the events with a golden file. After an intended change to a plugin's output, `go test ./plugins/ -update` rewrites its golden files; review that diff like code. - -`GITHUB_TOKEN` (optional, no scopes) only raises GitHub's rate limit when the `openapi` adapters look up release tags; the tests never use it. Nothing else is read from the environment. - -## Releases - -Releases are `v0.x.y` tags on `main` with a [GitHub release](https://github.com/patch-it-dev/patch-it-plugins/releases) and generated notes. The **Release** workflow tags a patch release on its own every Monday and Thursday at 14:00 UTC when something new was merged, and maintainers can run it by hand ([how](CONTRIBUTING.md#releasing)). patch-it's hosted ingestion service runs the latest release, so a merged plugin is running within a few days. Tags are never moved or deleted. - -```sh -go get github.com/patch-it-dev/patch-it-plugins@latest -``` - -## Contributing - -New providers and fixes are welcome: [CONTRIBUTING.md](CONTRIBUTING.md) has the rules (DCO sign-off, golden tests, the plugin list), [docs/writing-a-plugin.md](docs/writing-a-plugin.md) the how-to. AI coding agents: [AGENTS.md](AGENTS.md). Questions and ideas: [Discussions](https://github.com/patch-it-dev/patch-it-plugins/discussions). - -## License - -[Apache-2.0](LICENSE), copyright patch-it contributors. +You need Go (the version in `go.mod`), `make` and Python 3. + +1. **Scaffold it**, on the kind that reads its source ([which kind](docs/writing-a-plugin.md#3-pick-the-kind)): + ```sh + go run ./cmd/new-plugin --api acme --provider Acme --format feed --source https://acme.example/changelog.xml + ``` +2. **Fill in every `TODO(new-plugin)`**: the reader, the provider and source metadata in its `plugin.RegisterProvider` block, a saved copy of the source in `testdata/`, and "Why this source" in `providers/acme.md`. Then write the golden file and read it like code: + ```sh + go test ./plugins/feed -run TestAcme -update + ``` +3. **Check it the way CI will, and open the PR** (`git commit -s`, one provider per PR): + ```sh + make check + ``` + +## More + +- [How to write a plugin](docs/writing-a-plugin.md): providers, sources, kinds, versions, the test kit, step by step. +- [How it works](docs/how-it-works.md): what a plugin does, the layout, the checks, releases. +- What's covered: `make plugins` prints every provider and source from the code; each [release](https://github.com/patch-it-dev/patch-it-plugins/releases) carries it as `plugins.json`, and [docs.patch-it.dev](https://docs.patch-it.dev/supported-apis/plugins/) shows it. +- [CONTRIBUTING.md](CONTRIBUTING.md) (the rules), [AGENTS.md](AGENTS.md) (for AI coding agents), [Discussions](https://github.com/patch-it-dev/patch-it-plugins/discussions), [good first issues](https://github.com/patch-it-dev/patch-it-plugins/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22). +- What patch-it does with the events: [patch-it.dev](https://patch-it.dev) and [docs.patch-it.dev](https://docs.patch-it.dev). [Apache-2.0](LICENSE), copyright patch-it contributors. diff --git a/docs/README-template.md b/docs/README-template.md index e5fd3b8..53e37dd 100644 --- a/docs/README-template.md +++ b/docs/README-template.md @@ -4,15 +4,16 @@ Every public patch-it repo's README has the same shape, so a developer landing o ## The sections, in order +A README is **one screen**. Everything else goes in `docs/` and is linked. + | # | Section | What goes in it | |---|---|---| -| 1 | **What** (one line, bold, under the title) | What *this repo* is, in one sentence, in the product's words. For a part of patch-it, say which part of the story it is ("how patch-it notices, on its own, that something changed"). | -| 2 | **Why** (one short paragraph) | The one-liner for patch-it: when anything your code depends on changes (APIs, packages, SDKs, runtimes, CI, AI models), patch-it finds the code that breaks and opens the pull request that fixes it. In your CI, on your key. "You get the fix, not the alert." Then one sentence on why this repo matters to that. | -| 3 | **Badges** (one line of them) | The badge set below, in that order. | -| 4 | **Visual** | One picture of the flow: a mermaid diagram (GitHub renders it) until there's a GIF or screenshot. Left to right, ending in "pull request in your repo". | -| 5 | **The story** ("What patch-it does") | In this order, one or two lines each: (1) **it notices on its own** (it watches the change itself; no one assigns an alert; also changes with no version bump, like a model or API shut down on a date); (2) **it writes the fix** (an AI model on your key writes the code change, your CI runs your tests on the pull request like any PR, you review and merge it; never say patch-it tests the fix); (3) **proof** ("8 pull requests merged by outside maintainers", no other counts); (4) **layers covered**, as a table with two status columns, **Noticed** (a plugin watches it) and **Fixed** (patch-it writes the PR), each `live`, `in testing`, `building` or `planned`, never ahead of the truth; (5) **how to start** (a link to docs.patch-it.dev). | -| 6 | **Quickstart** (three or four steps, one screen) | The shortest path from a clean clone to the repo's main job, each step a command that runs as written, with the output to expect. Details live in linked docs. | -| 7 | **How it works, and links** | A short "how it works", the layout if the repo has code, how to run its checks, releases, contributing (with Discussions and good first issues), license, and docs.patch-it.dev. | +| 1 | **What** (two lines at most, bold first sentence, under the title) | What *this repo* is, in the product's words. For a part of patch-it, say which part of the story it is ("how patch-it notices, on its own, that something changed"). | +| 2 | **Badges** (one line of them) | The badge set below, in that order. | +| 3 | **Quickstart** (three steps) | The shortest path from a clean clone to the repo's main job, each step a command that runs as written. | +| 4 | **Links** | `docs/` pages (how-to, how it works), CONTRIBUTING, AGENTS, Discussions, good first issues, license, docs.patch-it.dev. | + +**Never a list of what's covered** (providers, plugins, layers with statuses, counts): those change at run time and the code is their only source. Say how to find them (the docs site, `make plugins`) instead. ## The badge set @@ -48,52 +49,17 @@ Use only images that are already public, linked by URL, never copied in: Patchy # -**** - -**Why:** when anything your code depends on changes (APIs, packages, SDKs, runtimes, CI, AI models), [patch-it](https://patch-it.dev) finds the code that breaks and opens the pull request that fixes it. In your CI, on your key. You get the fix, not the alert. +**** -```mermaid -flowchart LR - A["A dependency changes"] --> B[""] --> C["Pull request in your repo"] -``` - -## What patch-it does - -1. **It notices on its own.** ... -2. **It writes the fix.** ... -3. **Proof:** 8 pull requests opened by the early version have been merged by outside maintainers. -4. **What it covers:** - - | Layer | Noticed | Fixed | - |---|---|---| - | APIs | live | live | - | AI models | live | live | - | Packages and SDKs | live | in testing | - | Runtimes | live | building | - | CI | live | planned | +## -5. **Start:** [docs.patch-it.dev](https://docs.patch-it.dev). +1. ... +2. ... +3. ... -## +## More -## How it works - -## Contributing - -## License +- ```` - -Keep the layer statuses the same in every repo, and change them everywhere at once when one moves. - -## When a repo goes public - -The README is half of it. The same day, in the repo's settings: - -1. **Description**: one line in the same words (what the repo is, the layers, "You get the fix, not the alert"), under 350 characters, no version-bot names, never API-only. -2. **Website**: `https://docs.patch-it.dev` (or the page in the docs about this repo). -3. **Topics**: from `dependencies`, `breaking-changes`, `api-changes`, `api`, `sdk`, `ai-models`, `ci`, `pull-requests`, `automation`, plus the repo's language and what it is (`go`, `plugins`, `documentation`, …). -4. **Discussions** on, if people will ask questions there; keep the categories Q&A, Ideas, Show and tell and Announcements. -5. **Social preview** (Settings → General → Social preview; GitHub has no API for it): a 1280×640 PNG, dark background from the brand tokens, the plain wordmark, Patchy on a cream tile, IBM Plex, the repo's one line. No gradients or glows. -6. Check a shared link to the repo shows its own card. diff --git a/docs/README.md b/docs/README.md index be3cdc9..f5b8552 100644 --- a/docs/README.md +++ b/docs/README.md @@ -3,6 +3,7 @@ | Page | What's in it | |---|---| | [plugins.schema.json](plugins.schema.json) | The shape of `plugins.json`, the list of providers and sources that `go run ./cmd/list-plugins` prints from the registry and each release attaches. No list is committed. | +| [how-it-works.md](how-it-works.md) | Providers, sources and kinds; what a plugin does, version by version; the layout, the checks, releases. | | [README-template.md](README-template.md) | The shape every public patch-it repo's README follows: sections, badge set, words, images. | | [writing-a-plugin.md](writing-a-plugin.md) | How to add a provider: the `Plugin` interface, fetching through `env.Get` and `kit/`, saving the source, the golden test with `kittest`, the provider page and the source's metadata in its registration. | diff --git a/docs/how-it-works.md b/docs/how-it-works.md new file mode 100644 index 0000000..9a30707 --- /dev/null +++ b/docs/how-it-works.md @@ -0,0 +1,51 @@ +# How it works + +A **provider** (Stripe, OpenAI, GitHub…) has one category and one or more **sources**: each one place it announces changes (an OpenAPI spec repo, a changelog, a deprecations page, a feed), read by one **kind** of plugin (`plugins//`), at one **layer** of a customer's stack. A source's id is `provider/source` (`stripe/api`, `openai/models`); its old slug (`stripe`, `openai-models`) is what events carry in `api` until patch-it's change-event contract switches to source ids. Every provider and source is registered in code with its metadata (`plugin.RegisterProvider`): that registration is the only list. + +## What a plugin does + +0. **List the versions.** A source with versions (a spec's releases, a package on PyPI or npm, a guide per major) lists every one, oldest first (`ListVersions`, from `core/versions`: GitHub releases, tags and commits, PyPI, npm, the Web Archive at a date). Each consecutive pair is read and compared once (`ReadAt`, `Extract`), so a customer several versions behind gets every step of the upgrade. A living page with no versions (a deprecations page, a feed) is read as it is now; each change to it is a dated version. +1. **Fetch.** The plugin asks its environment for the source (`env.Get`); it never opens connections itself. Pages that change in place are also cited through a Web Archive snapshot, so the link in an event keeps showing what the plugin read. +2. **Compare.** It compares two versions of the source: two OpenAPI specs, two versions of a deprecations page, a release's changelog section against the release before. `Run` does the newest version against the last one it saw (the baseline); `Extract` does any pair. +3. **Emit events.** Each change becomes one event in [patch-it's change-event format](https://github.com/patch-it-dev/patch-it/tree/main/docs/contract): what changed, how bad it is (breaking, deprecation, behavior change, non-breaking), the date it takes effect, and the source URL with a quote. **No source, no event.** When unsure, a plugin emits nothing: a wrong event would become a wrong pull request. + +Most providers are an entry for an existing kind, not new code. patch-it's hosted ingestion service runs every source on a schedule and serves the events to the patch-it GitHub Action and CLI. + +## What's covered + +There's no list in this repo: `make plugins` prints every provider and source from the registry as JSON, every release attaches it as `plugins.json`, and [docs.patch-it.dev](https://docs.patch-it.dev/supported-apis/plugins/) shows it. + +## Layout + +| Path | What's there | +|---|---| +| [`core/plugin/`](../core/plugin/) | The `Plugin` interface (`Run`; `ListVersions`, `ReadAt`, `Extract`), the registry of providers and sources (`RegisterProvider`, `Lookup`, `LookupSource`, `Providers`), `Env`, `Result` | +| [`core/versions/`](../core/versions/) | Every version of a source: GitHub releases, tags and commits, PyPI, npm, the Web Archive at a date | +| [`plugins/`](../plugins/) | One package per kind, with its providers; `plugins/all` imports every one | +| [`cmd/new-plugin/`](../cmd/new-plugin/), [`cmd/list-plugins/`](../cmd/list-plugins/) | The starter for a new provider; the list of providers and sources, printed from the registry | +| [`core/kit/`](../core/kit/), [`core/kit/kittest/`](../core/kit/kittest/), [`core/github/`](../core/github/) | What plugins share (the unchanged-page check, Wayback citing, the newest release or commit on GitHub) and the test kit (in-memory store, saved pages, golden files) | +| [`core/event/`](../core/event/), [`core/migrate/`](../core/migrate/) | The change-event types of [patch-it's contract](https://github.com/patch-it-dev/patch-it/tree/main/docs/contract), and the v1 → v2 conversion | +| [`core/diff/`](../core/diff/), [`core/normalize/`](../core/normalize/), [`core/hint/`](../core/hint/), [`plugins/openapi/adapter/`](../plugins/openapi/adapter/) | The OpenAPI diff engine: compare two specs, turn the changes into events, attach changelog hints; one adapter per spec provider | +| [`plugins/retirements/retire/`](../plugins/retirements/retire/), [`plugins/migrationguide/sdkguide/`](../plugins/migrationguide/sdkguide/), [`core/fetch/`](../core/fetch/) | The deprecations-page and migration-guide readers; the HTTP fetch and cache | +| [`contract/fixtures/`](../contract/) | A copy of patch-it's change-event fixtures | +| `plugin/`, `event/`, `fetch/`, `migrate/`, `kit/`, `kit/kittest/`, `retire/`, `sdkguide/` | Deprecated: the old import paths, forwarding to the packages above until code that imports them moves | +| [`providers/`](../providers/) | One page per provider: why that source, what the plugin reads | +| [`docs/`](./) | [How to write a plugin](writing-a-plugin.md), the [README template](README-template.md) patch-it's public repos share | + +## Run the checks + +```sh +make check # what CI runs: tidy, build, gofmt, vet, import and net/http guards, go test -race, no TODO(new-plugin) left, leak guard +``` + +The tests are offline: every plugin's test reads a saved copy of its source from `testdata/` and compares the events with a golden file. After an intended change to a plugin's output, `go test ./plugins/ -update` rewrites its golden files; review that diff like code. + +`GITHUB_TOKEN` (optional, no scopes) only raises GitHub's rate limit when the `openapi` adapters look up release tags; the tests never use it. Nothing else is read from the environment. + +## Releases + +Releases are `v0.x.y` tags on `main` with a [GitHub release](https://github.com/patch-it-dev/patch-it-plugins/releases) and generated notes. The **Release** workflow tags a patch release on its own every Monday and Thursday at 14:00 UTC when something new was merged, and maintainers can run it by hand ([how](../CONTRIBUTING.md#releasing)). patch-it's hosted ingestion service runs the latest release, so a merged plugin is running within a few days. Each release carries `plugins.json`, the list of providers and sources printed from the registry at that tag. Tags are never moved or deleted. + +```sh +go get github.com/patch-it-dev/patch-it-plugins@latest +``` diff --git a/docs/writing-a-plugin.md b/docs/writing-a-plugin.md index 4a19b9e..58aab2b 100644 --- a/docs/writing-a-plugin.md +++ b/docs/writing-a-plugin.md @@ -1,18 +1,18 @@ # Writing a plugin -A plugin watches one provider source and turns what changed into change events. This guide walks through adding one, from picking a source to the PR. The rules are in [CONTRIBUTING.md](../CONTRIBUTING.md); this is the how-to. +A **provider** (Stripe, OpenAI, Kubernetes…) has a **category** and one or more **sources**. A source is one place the provider announces changes; it is read by one **kind** of plugin (the `plugins/` package: `openapi`, `changelog`, `feed`, …), covers one **layer** (`api`, `sdk`, `package`, `runtime`, `ci`, `ai-models`, `infra`), and has **versions**. Its id is `provider/source` (`stripe/api`, `algolia/search-api`). This guide walks through adding a source, from picking it to the PR. The rules are in [CONTRIBUTING.md](../CONTRIBUTING.md); this is the how-to. How the pieces fit: [how-it-works.md](how-it-works.md). -**Most new providers don't need new code.** They are a new entry for an existing format plugin: a new `changelog.File`, a new `retire.Page`, a new `feed.Feed` reader, a new OpenAPI adapter. Write a new format plugin only when none of them reads your source. +**Most new sources don't need new code.** They are a new entry for an existing kind: a new `changelog.File`, a new `retire.Page`, a new `feed.Feed` reader, a new OpenAPI adapter. Write a new kind only when none of them reads your source. There is no "Kubernetes kind" or "Terraform kind": Kubernetes API removals are a migration-guide or notices source, a client SDK is a package source, a Terraform provider is a changelog source. ## 1. Scaffold it with `new-plugin` -Pick the source and the format plugin that reads it (steps 2 and 3), then let the starter write everything around the provider: +Pick the source and the kind that reads it (steps 2 and 3), then let the starter write everything around the provider: ```sh go run ./cmd/new-plugin --api acme --provider Acme --format openapi # --format: openapi, retirements, migrationguide, removedmembers, changelog, feed, apichangelog, catalog -# optional: --source https://... (the provider's source), --watches "...", --layer API|SDK|package|runtime|CI|"AI models", -# --page acme (a second plugin of a provider shares its page in providers/) +# optional: --source https://... (the provider's source), --watches "...", --layer api|sdk|package|runtime|ci|ai-models|infra, +# --page acme (a second source of a provider shares its page in providers/) ``` It writes, and lists: @@ -59,7 +59,7 @@ Then check what's left against the provider's own words: Write the numbers on the provider page, as `providers/vercel.md` does for its spec's history, so a reviewer can check the rule against them. -### Pin, and pick a version a repo pins +### Cite a version, and pick one a repo pins - **Cite what never changes:** a release tag, or the commit SHA of the release or of a branch's head. A moving branch or a live URL isn't a citation. - **"Newest" needs care:** @@ -70,15 +70,15 @@ Write the numbers on the provider page, as `providers/vercel.md` does for its sp - The patch-it client gates a package version as a package. It keeps a date-versioned API's events only if the API is in its list of date-versioned APIs, so **a new date-versioned API needs an entry there too**. - Two specs of one version (fixes on a release branch) aren't a release: `RemovalsNeedANewVersion` drops removals between them. -## 3. Pick the format plugin +## 3. Pick the kind -| Format plugin | Reads | Add a provider by | +| Kind | Reads | Add a source by | |---|---|---| -| `plugins/openapi` | An OpenAPI spec per release, diffed with `core/diff/` and turned into events by `core/normalize/` | An `adapter.Adapter` in `plugins/openapi/adapter/` (latest tag, fetch a tag's spec) and a `plugin.Register(Plugin{Adapter: …})` line | +| `plugins/openapi` | An OpenAPI spec per release, diffed with `core/diff/` and turned into events by `core/normalize/` | An `adapter.Adapter` in `plugins/openapi/adapter/` (latest tag, fetch a tag's spec; `Versions` to list every release) and a `plugin.RegisterProvider` block | | `plugins/retirements` | A model provider's deprecations page (a table of date, model, replacement) | A `retire.Page` in `plugins/retirements/retire/providers.go` | -| `plugins/migrationguide` | An SDK's migration guide at a release tag (removed request parameters) | A `sdkguide.Guide` in `plugins/migrationguide/sdkguide/` | +| `plugins/migrationguide` | An SDK's migration guide at the tag of each new major (removed request parameters) | A `sdkguide.Guide` in `plugins/migrationguide/sdkguide/` | | `plugins/removedmembers` | A package's table of removed members at a release tag (an RST simple table or a Markdown pipe table) | A `removedmembers.Guide` | -| `plugins/changelog` | A `CHANGELOG.md` / `CHANGES.rst` / `HISTORY.rst` at a release tag | A `changelog.File` | +| `plugins/changelog` | A `CHANGELOG.md` / `CHANGES.rst` / `HISTORY.rst`, each release's section at its tag (releases from PyPI or npm) | A `changelog.File` with its `TagPrefix` | | `plugins/feed` | An RSS or Atom feed | A `feed.Feed` with a reader for that vendor's entries | | `plugins/catalog` | A flat JSON catalog, one entry per name, with no release tags to pin — a commit SHA stands in | A `catalog.Catalog`, diffed between two snapshots for names that disappeared entirely | | `plugins/apichangelog` | A vendor's machine-readable API changelog in JSON (per day, endpoint and API version, oasdiff change codes), at the commit that last changed it | An `apichangelog.Log` | @@ -90,13 +90,8 @@ A plugin implements `plugin.Plugin` (package [`core/plugin/`](../core/plugin/plu ```go type Plugin interface { - Name() string // "stripe": the --api id and every event's `api` - Run(ctx context.Context, env Env) (Result, error) // what changed since the baseline it saved last -} - -// Optional: diff two given releases, to recreate a historical change. -type Replayer interface { - Replay(ctx context.Context, env Env, from, to string) (Result, error) + Name() string // the source's old slug: --api and every event's `api` + Run(ctx context.Context, env Env) (Result, error) // the newest version against the baseline it saved last } type Env struct { @@ -113,29 +108,57 @@ type Result struct { } ``` -A plugin **stores nothing**. It reads its source, compares it with its last baseline (`env.Baselines.LastSpec(api)`), and returns events and the new baseline. Whoever runs it (patch-it's hosted ingestion service, or the test kit) stores the events first and the baseline after, so a crash in between re-reads the same change next time and the fingerprints dedupe it. +A plugin **stores nothing**. Whoever runs it (patch-it's hosted ingestion service, or the test kit) stores the events first and the baseline after, so a crash in between re-reads the same change next time and the fingerprints dedupe it. + +### Every version, not only the newest + +A source that has versions (a spec's releases, a package's releases, a guide per major) also implements the three parts of `plugin.Versioned` ([`core/plugin/versions.go`](../core/plugin/versions.go)), so a customer several versions behind gets every step: + +```go +ListVersions(ctx, env) ([]Version, error) // every version, oldest first +ReadAt(ctx, env, v Version) (Snapshot, error) // the source at one version +Extract(ctx, env, from, to Snapshot) (Result, error) // the changes from one to the next +``` + +- **List** with [`core/versions`](../core/versions/versions.go): `GitHubReleases`, `GitHubTags`, `GitHubCommits` (the commits that changed a file), `PyPIReleases`, `NPMReleases`, `WaybackAt` (a page as it was on a date). Never pin a tag in code: a version you need for a golden test is a pinned *case* in the test (`changelog.Urllib3.At("2.0.0")`), not the registered source. +- **Read at** a version's `Ref` (a tag, a commit SHA, an archive URL) and return the exact URL read as the snapshot's `SourceURL`: it's the events' citation. +- **Extract** sets `previous_version` to `from`'s version; `from` may be zero (nothing before). It returns no baselines. +- `Run` is built from the same parts: the newest version, extracted against the last baseline. + +A living page with no versions of its own (a deprecations page, a feed) only has `Run`: it's read as it is now, and each change to it is a dated version. + +### Register the provider and its source -Register the plugin in its package's `init`: +In the kind's package `init`, with the metadata ([AGENTS.md](../AGENTS.md) rule 1): ```go func init() { - plugin.Register(Plugin{File: Textual0860}) + plugin.RegisterProvider(plugin.Provider{ + ID: "urllib3", Name: "urllib3", Category: plugin.CategoryPackages, Homepage: "https://urllib3.readthedocs.io", + Sources: []plugin.Source{{ + ID: "urllib3", Layer: plugin.LayerPackage, Page: "providers/urllib3.md", + Watches: "Names removed or deprecated in each release, …", + URLs: []string{"https://github.com/urllib3/urllib3/blob/main/CHANGES.rst"}, + Plugin: Plugin{File: Urllib3}, + }}, + }) } ``` -`Register` panics on a duplicate name or the name `all`. +`RegisterProvider` panics on missing metadata, a duplicate id, or a provider registered twice with a different name, category or homepage (a provider with sources in two kinds repeats the same ones). `plugin.Lookup` and `plugin.LookupSource` find a source by `provider/source` or by its old slug. ### Fetching: `env.Get` and `core/kit/` - Fetch **only** with `env.Get`. It caches by `ETag` / `Last-Modified`, retries, and in tests serves saved files. `CI_go` fails a package outside `core/fetch/`, `core/kit/` and `plugins/openapi/adapter/` that imports `net/http`. - `kit.Unchanged(env, api, body)`: true when the page hashes to the last baseline. Return early: an unchanged source costs one GET and nothing else. - `kit.Cite(ctx, archiver, url, read, want)`: for a **live page**. Takes a Wayback Machine snapshot, reads it with your own reader and checks it says exactly what the live page said. Make the events from the snapshot and cite its URL. No snapshot, no events. -- `plugin.Baseline(api, version, tag, sourceURL, content, replay, now)`: the baseline to return. -- Older format plugins build v1 `event.ChangeEvent`s and convert them with `migrate.Fresh`; new code builds `event.EventV2` directly. +- `plugin.Baseline(api, version, tag, sourceURL, content, replay, now)`: the baseline `Run` returns. +- `github.LatestRelease`, `github.LatestCommit`, `github.Commit`, `github.Head` ([`core/github`](../core/github/github.go)): the newest release or commit on GitHub, one place; don't decode GitHub's responses yourself. +- Older kinds build v1 `event.ChangeEvent`s and convert them with `migrate.Fresh`; new code builds `event.EventV2` directly. ### Events -The event format is [patch-it's change-event contract](https://github.com/patch-it-dev/patch-it/tree/main/docs/contract); `event/` is its Go binding. Every event has a typed `subject` (`api_operation`, `model`, `package_release`, `ci_runtime`), a `change_type` (`breaking`, `deprecation`, …), a `version`, a `fingerprint` (`event.FingerprintV2`), and **a `source_url` and `source_excerpt`: no source, no event**. Call `Validate()`; an invalid event goes in `Result.Rejected`, not `Events`. +The event format is [patch-it's change-event contract](https://github.com/patch-it-dev/patch-it/tree/main/docs/contract); `core/event/` is its Go binding. Every event has a typed `subject` (`api_operation`, `model`, `package_release`, `ci_runtime`), a `change_type` (`breaking`, `deprecation`, …), a `version`, a `fingerprint` (`event.FingerprintV2`), and **a `source_url` and `source_excerpt`: no source, no event**. Call `Validate()`; an invalid event goes in `Result.Rejected`, not `Events`. ## 5. Save the source @@ -168,7 +191,7 @@ func TestGolden(t *testing.T) { - **One drop test**: a realistic input the plugin must *not* turn into an event (an unrelated section, a behavior change written as prose, a product name in a sentence). Precision over recall: a missed change is a bug we fix later; a wrong change is a wrong pull request in someone's repo. Make it prove something: also run the same input with the filter off, and assert the noise it drops is there. - **`registered_test.go`**: `kittest.Registered(t, Plugin{…})` checks the package registered that plugin under that name. -- **A new format plugin package**: add its blank import to [`plugins/all/all.go`](../plugins/all/all.go). `TestEveryPluginIsImported` fails until you do; the hosted service runs exactly what `plugins/all` imports. +- **A new kind (a new package under plugins/)**: add its blank import to [`plugins/all/all.go`](../plugins/all/all.go). `TestEveryPluginIsImported` fails until you do; the hosted service runs exactly what `plugins/all` imports. ## 8. Write the provider page