From 330a8ed69ef4b9d68002968069d4847a6c282f0b Mon Sep 17 00:00:00 2001 From: adssib <75389300+adssib@users.noreply.github.com> Date: Fri, 20 Nov 2026 16:51:25 -0500 Subject: [PATCH] print the plugin list from the registry, commit none docs/plugins.md and docs/plugins.json are gone, with the README's coverage table and plugin-count badge and the providers/ page table. The registry is the only list: go run ./cmd/list-plugins (make plugins) prints every provider and source with its metadata as JSON (docs/plugins.schema.json), and the Release workflow attaches that output to each release as plugins.json. AGENTS.md rule 1 is now: every new or changed provider or source has complete metadata in its registration, and a providers/ page. new-plugin no longer writes a table row; the PR template and the review prompts check the registration instead. Co-Authored-By: Claude Opus 5.5 Signed-off-by: adssib <75389300+adssib@users.noreply.github.com> --- .github/pull_request_template.md | 4 +- .github/workflows/CI_go.yml | 1 - .github/workflows/claude-code-review.yml | 12 +- .github/workflows/claude.yml | 2 +- .github/workflows/release.yml | 11 +- AGENTS.md | 17 +- CONTRIBUTING.md | 8 +- Makefile | 13 +- README.md | 23 +- cmd/list-plugins/main.go | 53 +--- cmd/new-plugin/formats.go | 12 +- cmd/new-plugin/main.go | 67 ++--- cmd/new-plugin/main_test.go | 26 +- docs/README-template.md | 2 +- docs/README.md | 5 +- docs/plugins.json | 307 ----------------------- docs/plugins.md | 56 ----- docs/plugins.schema.json | 47 ++-- docs/writing-a-plugin.md | 9 +- internal/pluginlist/pluginlist.go | 280 +++++---------------- internal/pluginlist/pluginlist_test.go | 133 ++++------ internal/pluginlist/schema_test.go | 85 ++++--- providers/README.md | 33 +-- 23 files changed, 296 insertions(+), 910 deletions(-) delete mode 100644 docs/plugins.json delete mode 100644 docs/plugins.md diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index 020aa01..5e14f73 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -4,14 +4,14 @@ ## Checklist -- [ ] **Added to / updated in `docs/plugins.md`** (new plugin: a row; changed id, provider, source, what it watches, format or status: the row updated; removed: the row removed). Not applicable: +- [ ] **Metadata complete in the registration** (`plugin.RegisterProvider`: provider id, name, category, homepage; each source's id, layer, what it watches, URLs, page), and I read the words. There is no list to update. Not applicable: - [ ] Every commit is signed off (`git commit -s`, DCO) - [ ] Golden test over a real saved copy of the source (`testdata/golden/.json`) - [ ] `testdata/README.md` lists each file: URL, date fetched, license, quote of the part the plugin reads - [ ] One pinned drop test: a realistic input that produces no event - [ ] `providers/.md` written or updated from `providers/_template.md`, including **Why this source** - [ ] Fetches only through `env.Get` / `core/kit/` -- [ ] `make check` passes locally (and `make plugins-json` was run if a plugin or its row changed) +- [ ] `make check` passes locally (`make plugins` prints the source as you expect) - [ ] Nothing private: no credentials, internal hosts, private links, customer names ## Notes for the reviewer diff --git a/.github/workflows/CI_go.yml b/.github/workflows/CI_go.yml index 315ed6f..82162c1 100644 --- a/.github/workflows/CI_go.yml +++ b/.github/workflows/CI_go.yml @@ -80,7 +80,6 @@ jobs: run: make todos # -race: the fetcher and the tests' HTTP servers run on other goroutines. - # Also fails when docs/plugins.json is out of date (make plugins-json). - name: go test if: steps.go.outputs.present == 'true' run: make test diff --git a/.github/workflows/claude-code-review.yml b/.github/workflows/claude-code-review.yml index 6f33f2f..cb7b4f9 100644 --- a/.github/workflows/claude-code-review.yml +++ b/.github/workflows/claude-code-review.yml @@ -93,12 +93,12 @@ jobs: any issue it references with `gh issue view `. Read the diff with `gh pr diff `, and changed files in full where the diff isn't enough context. - 2. THE PLUGIN LIST (AGENTS.md Rule 1). Every plugin added, removed, or with a - changed id, provider, source, what it watches, format plugin or status has - its row added, removed or updated in docs/plugins.md, between the - `` and `` markers, as a plain - Markdown table with absolute links only. Compare the registered plugins with - the table yourself; a missing row is blocking. + 2. THE METADATA (AGENTS.md Rule 1). Every source added or changed is + registered with plugin.RegisterProvider with complete, accurate metadata: + the provider's id, name, category and homepage; each source's id, layer, + what it watches, the URLs it reads and its page. There is no hand-kept list + (no docs/plugins.md, no plugins.json in the repo): one reappearing is + blocking. 3. PROVIDER PAGE. Every new provider has providers/.md copied from providers/_template.md, with a real **Why this source**: the link, the date it was checked, why it's the best source, the alternatives not used, and a diff --git a/.github/workflows/claude.yml b/.github/workflows/claude.yml index aabd647..3040e8f 100644 --- a/.github/workflows/claude.yml +++ b/.github/workflows/claude.yml @@ -68,4 +68,4 @@ jobs: --permission-mode default --allowedTools "mcp__github_inline_comment__create_inline_comment,Read,Grep,Glob,Bash(gh pr view:*),Bash(gh pr diff:*),Bash(gh issue view:*)" --disallowedTools "Edit,Write,MultiEdit,NotebookEdit,WebFetch,WebSearch,Bash(git push:*),Bash(git commit:*),Bash(git add:*),Bash(git rm:*),mcp__github_file_ops__commit_files,mcp__github_file_ops__delete_files" - --append-system-prompt "This is patch-it-plugins, a public repo of source plugins. Everything you write is public. You can only read and comment; you can't change files or run code, so if asked to, say so and describe the change instead. When asked to review a pull request (for example '@claude review'), check it against AGENTS.md: (1) every new, changed or removed plugin has its row added, updated or removed in docs/plugins.md between the plugins:start and plugins:end markers, absolute links only; (2) every new provider has providers/.md from providers/_template.md with a real 'Why this source' (link, date checked, why it's best, alternatives, quote); (3) a golden test over a real saved source, a testdata/README.md row (URL, date fetched, license, quote), one pinned drop test, every event carries its source URL and a quote, and golden edits are explained; (4) fetch only through env.Get or core/kit/, no net/http outside core/fetch/, core/kit/ and plugins/openapi/adapter/, no filesystem outside testdata/; (5) precision over recall: flag anything that could emit an event for a non-change; (6) nothing private: no credentials, internal hostnames or service names, links to private repos or tickets, customer names, or links to AI sessions. You can't run the tests; say that CI_go covers them. Start with one verdict line ('✅ Ready to merge', '⚠️ Merge after fixes' or '❌ Not ready'), then a table of rule | status | evidence, blocking bugs, and clearly marked non-blocking suggestions. Be kind and specific: many authors are outside contributors. Never include a Claude session ID or claude.ai link." + --append-system-prompt "This is patch-it-plugins, a public repo of source plugins. Everything you write is public. You can only read and comment; you can't change files or run code, so if asked to, say so and describe the change instead. When asked to review a pull request (for example '@claude review'), check it against AGENTS.md: (1) every new or changed source is registered with plugin.RegisterProvider with complete, accurate metadata (provider id, name, category, homepage; source id, layer, what it watches, URLs, page), and no hand-kept list of plugins comes back; (2) every new provider has providers/.md from providers/_template.md with a real 'Why this source' (link, date checked, why it's best, alternatives, quote); (3) a golden test over a real saved source, a testdata/README.md row (URL, date fetched, license, quote), one pinned drop test, every event carries its source URL and a quote, and golden edits are explained; (4) fetch only through env.Get or core/kit/, no net/http outside core/fetch/, core/kit/ and plugins/openapi/adapter/, no filesystem outside testdata/; (5) precision over recall: flag anything that could emit an event for a non-change; (6) nothing private: no credentials, internal hostnames or service names, links to private repos or tickets, customer names, or links to AI sessions. You can't run the tests; say that CI_go covers them. Start with one verdict line ('✅ Ready to merge', '⚠️ Merge after fixes' or '❌ Not ready'), then a table of rule | status | evidence, blocking bugs, and clearly marked non-blocking suggestions. Be kind and specific: many authors are outside contributors. Never include a Claude session ID or claude.ai link." diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 8e1d14c..5cc4f74 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -172,9 +172,18 @@ jobs: git tag -a "$TAG" "$SHA" -m "$TAG" -m "Released by the Release workflow ($by)." git push origin "refs/tags/$TAG" + # The list of providers and sources, from the registry at the tag: the + # release's plugins.json asset. Never committed; the docs site reads the + # latest release's copy. + - name: list the providers and sources + if: steps.version.outputs.skip != 'true' + env: + TAG: ${{ steps.version.outputs.tag }} + run: go run ./cmd/list-plugins -ref "$TAG" -o "$RUNNER_TEMP/plugins.json" + - name: publish the GitHub release if: steps.version.outputs.skip != 'true' env: GH_TOKEN: ${{ github.token }} TAG: ${{ steps.version.outputs.tag }} - run: gh release create "$TAG" --verify-tag --generate-notes --title "$TAG" + run: gh release create "$TAG" --verify-tag --generate-notes --title "$TAG" "$RUNNER_TEMP/plugins.json" diff --git a/AGENTS.md b/AGENTS.md index 632c195..7d832cb 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -2,13 +2,11 @@ Rules for AI coding agents (and humans) working in this repo. `CLAUDE.md` imports this file. -## Rule 1: every new or changed plugin goes in `docs/plugins.md` +## Rule 1: every new or changed provider or source has complete metadata in its registration -**If you add a plugin, add its row to [`docs/plugins.md`](docs/plugins.md) in the same PR. If you change a plugin's id, provider, source, what it watches, format plugin or status, update its row in the same PR. If you remove one, remove its row.** +**A source exists when its kind's package registers it with `plugin.RegisterProvider`, and its metadata lives only there: the provider's id, name, category and homepage, and each source's id, layer, what it watches, the URLs it reads and its page, `providers/.md`.** Adding, changing or removing a source is a change to that registration (and to its page) in the same PR. There is no list to update: nothing in this repo, the docs site or anywhere else keeps one by hand. -The list of supported APIs on docs.patch-it.dev is generated from that file, so a plugin missing from it doesn't exist as far as users can tell. Then run `make plugins-json`: it regenerates [`docs/plugins.json`](docs/plugins.json) from the plugin registry and the table (and the plugin count on the README's badge), and fails if a registered plugin has no row (or a row has no plugin, a wrong format or no provider page). `go test` fails while `docs/plugins.json` is out of date. The check can't tell whether a row's words are right: you are that check. Say in the PR that you compared them. - -Keep the table between the `` and `` markers a plain Markdown table, one row per plugin, absolute links only (the docs site copies it verbatim). +`go run ./cmd/list-plugins` prints the list from the registry (JSON), and the Release workflow attaches it to every release as `plugins.json`; the docs site and patch-it's hosted ingestion service read it from there. `RegisterProvider` refuses incomplete metadata and `go test` fails when a source's page is missing, but no check can tell whether the words are right: you are that check. Say in the PR that you read them. ## Rule 2: this repo is public @@ -41,7 +39,7 @@ Refer to the service that runs these plugins as "patch-it's hosted ingestion ser ```sh make check # tidy, build, gofmt, vet, the import allowlist, the net/http guard, go test -race, no TODO(new-plugin) left, leak guard make check PATCH_IT=../patch-it # also CI_contract: contract/fixtures/ against a patch-it checkout -make plugins-json # regenerate docs/plugins.json after adding or changing a plugin or its row +make plugins # print the list of providers and sources from the registry (what each release attaches as plugins.json) ``` One target at a time: `make tidy`, `make build`, `make fmt`, `make vet`, `make imports`, `make nethttp`, `make test`, `make todos`, `make leak-guard`, `make contract`. Use `make GO=/path/to/go check` for another Go. @@ -63,11 +61,10 @@ plugins// one package per kind, with its providers: openapi (+ adap plugins/all imports every one plugin/ event/ fetch/ migrate/ kit/ retire/ sdkguide/ deprecated old import paths: they only forward to the packages above; don't add to them -cmd/list-plugins/ writes docs/plugins.json (make plugins-json); internal/pluginlist/ is its logic +cmd/list-plugins/ prints the list from the registry (make plugins); internal/pluginlist/ is its logic cmd/new-plugin/ scaffolds a provider on a format plugin (go run ./cmd/new-plugin --help) contract/fixtures/ a copy of patch-it's change-event fixtures (CI_contract checks it) -docs/plugins.md the list of supported plugins (Rule 1); docs/plugins.json is generated from it -Makefile make check (what CI_go runs, plus the leak guard), make plugins-json +Makefile make check (what CI_go runs, plus the leak guard), make plugins docs/ contributor docs, including writing-a-plugin.md providers/.md one page per provider scripts/ leak_guard.py (hashed internal names), check_contract.py @@ -81,7 +78,7 @@ The [change-event contract](https://github.com/patch-it-dev/patch-it/tree/main/d - Sign off every commit: `git commit -s` (DCO; `CI_dco` checks it). - Subjects short, lowercase, imperative: `add the foo plugin`. -- One plugin or fix per PR; fill in every checkbox of the PR template, including "added to docs/plugins.md". +- One plugin or fix per PR; fill in every checkbox of the PR template, including "metadata complete in the registration". - AI-assisted commits keep their co-author trailer. - Never re-run, dispatch or retrigger CI to "see if it passes"; fix the cause and push. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 7a2fde0..8554f02 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -2,11 +2,11 @@ Thanks for helping patch-it watch more APIs. This file is the rulebook; [docs/writing-a-plugin.md](docs/writing-a-plugin.md) is the how-to. -**Start a new provider with the starter:** `go run ./cmd/new-plugin --api --provider --format ` writes the definition, its registration, a test over placeholder sources, the testdata rows, the provider page and the `docs/plugins.md` row; you fill in the `TODO(new-plugin)`s ([how](docs/writing-a-plugin.md#1-scaffold-it-with-new-plugin)). +**Start a new provider with the starter:** `go run ./cmd/new-plugin --api --provider --format ` writes the definition, its registration, a test over placeholder sources, the testdata rows, the provider page, and the source's metadata in its `RegisterProvider` block; you fill in the `TODO(new-plugin)`s ([how](docs/writing-a-plugin.md#1-scaffold-it-with-new-plugin)). ## The rules -1. **Add your plugin to [docs/plugins.md](docs/plugins.md).** Every new plugin gets a row; every change to a plugin's id, provider, source, what it watches or status updates its row. The list on docs.patch-it.dev is built from that file, so a plugin that isn't listed is invisible to users. Then run `make plugins-json` to regenerate [`docs/plugins.json`](docs/plugins.json); it fails if the registry and the table disagree, and the tests fail while the JSON is out of date. +1. **Give every provider and source complete metadata in its registration.** A source exists when its kind's package registers it with `plugin.RegisterProvider`: the provider's id, name, category and homepage, and the source's id, layer, what it watches, the URLs it reads and its page. That registration is the only list: `make plugins` prints it, each release carries it as `plugins.json`, and docs.patch-it.dev reads it from there. Nothing else to update. 2. **Fetch only through the environment.** Plugins get their sources from `env.Get` or the helpers in `core/kit/`, never from `net/http` directly, the filesystem or a database. That keeps tests offline and lets the hosted service cache, retry and rate-limit politely. 3. **Every plugin has a golden test** over a real saved copy of its source: `testdata/` in, `testdata/golden/.json` out. `testdata/README.md` lists each file with its URL, the date you fetched it, its license, and a short quote of the part your plugin reads. Keep openly licensed files whole; trim vendor documentation pages to what the test needs. 4. **Precision over recall.** A missed change is a bug we can fix later. A wrong change becomes a wrong pull request in someone's repo. When in doubt, don't emit. @@ -36,7 +36,7 @@ That adds `Signed-off-by: Your Name `, matching the commit's au | Workflow | What it checks | |---|---| -| `CI_go` | `go mod tidy`, build, vet, gofmt, `go test -race` (including that `docs/plugins.json` is up to date); plugins import only this module, the standard library and approved dependencies; `net/http` only in the fetch layer. Each step is a [Makefile](Makefile) target: `make check` runs them all locally | +| `CI_go` | `go mod tidy`, build, vet, gofmt, `go test -race` (including that every registered source has its metadata and page); plugins import only this module, the standard library and approved dependencies; `net/http` only in the fetch layer. Each step is a [Makefile](Makefile) target: `make check` runs them all locally | | `CI_dco` | Every commit in the PR is signed off by its author | | `CI_leak_guard` | gitleaks over the PR's commits, and a check for known internal names (stored as hashes only). Also runs weekly over the whole repo | | `Claude Code Review` | Claude reviews the PR against the rules above, once (PRs from this repo's branches; forks: a maintainer comments `@claude review`). Advisory, not a required check | @@ -55,7 +55,7 @@ Releases are tags on `main` (`v0.x.y`), made only by the **Release** workflow. N - **patch** (`v0.4.1` → `v0.4.2`): fixes, a new provider on an existing format plugin, docs; - **minor** (`v0.4.1` → `v0.5.0`): a new format plugin, new exported API, anything users should read about. 3. The workflow refuses if `main`'s HEAD is already tagged, then runs, on that exact commit, the internal-names check over every file, gitleaks over the whole history, `go build`, `go vet`, `gofmt` and `go test -race`. Any failure stops it before a tag exists. -4. When everything passes it pushes an annotated tag and publishes a GitHub release with generated notes (grouped by the PR's label: `plugin`/`enhancement`, `bug`, `documentation`; see [`.github/release.yml`](.github/release.yml)). +4. When everything passes it pushes an annotated tag and publishes a GitHub release with generated notes (grouped by the PR's label: `plugin`/`enhancement`, `bug`, `documentation`; see [`.github/release.yml`](.github/release.yml)). The release carries one asset, `plugins.json`: the list of providers and sources printed from the registry at that tag (`go run ./cmd/list-plugins`). It is the only published list; nothing in the repo keeps one, and docs.patch-it.dev reads `https://github.com/patch-it-dev/patch-it-plugins/releases/latest/download/plugins.json`. `v1.0.0` (and any later major) is not a button: it is a deliberate decision a maintainer makes and tags by hand. diff --git a/Makefile b/Makefile index 75bd27f..5aa7bcd 100644 --- a/Makefile +++ b/Makefile @@ -5,7 +5,7 @@ # # make check everything below, in CI's order # make check PATCH_IT=../patch-it also compare contract/fixtures/ with patch-it's -# make plugins-json regenerate docs/plugins.json from the registry and docs/plugins.md +# make plugins print the providers and sources from the registry (JSON) # # The `::error` lines are GitHub annotations; locally they're plain messages. @@ -17,7 +17,7 @@ PYTHON ?= python3 MODULE := github.com/patch-it-dev/patch-it-plugins ALLOWED := ^($(MODULE)(/|$$)|gopkg\.in/yaml\.v3$$) -.PHONY: check tidy build fmt vet imports nethttp test todos leak-guard contract plugins-json +.PHONY: check tidy build fmt vet imports nethttp test todos leak-guard contract plugins check: tidy build fmt vet imports nethttp test todos leak-guard contract @echo "make check: all passed" @@ -58,13 +58,12 @@ nethttp: fi # -race: the fetcher and the tests' HTTP servers run on other goroutines. -# Includes the check that docs/plugins.json is up to date (internal/pluginlist). test: $(GO) test -race -count=1 ./... # A provider started with new-plugin is done when nothing it wrote still says # TODO(new-plugin). -TODO_DIRS := plugins core/migrate providers docs/plugins.md +TODO_DIRS := plugins core/migrate providers todos: @left=$$(grep -rl 'TODO(new-plugin)' $(TODO_DIRS) || true); \ if [ -n "$$left" ]; then \ @@ -87,5 +86,7 @@ contract: echo "contract: skipped (make check PATCH_IT= compares contract/fixtures/)"; \ fi -plugins-json: - $(GO) run ./cmd/list-plugins +# The list of what patch-it watches, from the registry. Never committed: the +# Release workflow attaches it to each release as plugins.json. +plugins: + @$(GO) run ./cmd/list-plugins diff --git a/README.md b/README.md index 870f92c..b021b0b 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,6 @@ [![Go Reference](https://pkg.go.dev/badge/github.com/patch-it-dev/patch-it-plugins.svg)](https://pkg.go.dev/github.com/patch-it-dev/patch-it-plugins) [![Go version](https://img.shields.io/github/go-mod/go-version/patch-it-dev/patch-it-plugins)](go.mod) [![License: Apache-2.0](https://img.shields.io/badge/license-Apache--2.0-3f5bf0)](LICENSE) -[![Plugins](https://img.shields.io/badge/plugins-30-3f5bf0)](docs/plugins.md) [![Docs](https://img.shields.io/badge/docs-docs.patch--it.dev-3f5bf0)](https://docs.patch-it.dev) [![Discussions](https://img.shields.io/github/discussions/patch-it-dev/patch-it-plugins)](https://github.com/patch-it-dev/patch-it-plugins/discussions) @@ -27,18 +26,7 @@ flowchart LR 1. **It notices on its own.** A plugin here sees the change where the provider announces it, and patch-it picks it up from there: no alert to triage, no ticket to assign. That includes changes that never bump a version, like an AI model or an API shut down on a date. 2. **It writes the fix.** An AI model (on your own key) reads the change and the lines it affects and writes the code change. It arrives as a pull request: your CI runs your tests on it, like any PR, and you review and merge it. The code change, not just the version. 3. **Proof:** 8 pull requests opened by the early version have been merged by outside maintainers. -4. **What it covers:** - - | Layer | Noticed (a plugin here watches it) | Fixed (patch-it writes the PR) | - |---|---|---| - | APIs: Stripe, GitHub, GitLab, Datadog, Vercel, Supabase, Algolia, MongoDB Atlas, Cloudflare | live | live | - | AI models: OpenAI and Anthropic model retirements | live | live | - | AI models: Vertex AI model retirements | live | planned | - | Packages and SDKs: NumPy, Textual, urllib3, the Anthropic Python SDK | live | in testing | - | Runtimes: Redis | live | building | - | CI: GitHub Actions runner retirements | live | planned | - - As of 2026-11-19; every plugin is in [docs/plugins.md](docs/plugins.md). +4. **What it covers:** APIs, AI models, packages and SDKs, runtimes and CI. The current list of providers and sources is generated from the code: see [docs.patch-it.dev](https://docs.patch-it.dev/supported-apis/plugins/), or run `make plugins`. 5. **Start:** [docs.patch-it.dev](https://docs.patch-it.dev) sets up the GitHub Action in your repo. To have it watch a provider it doesn't cover yet, add a plugin (below). @@ -54,7 +42,6 @@ go run ./cmd/new-plugin --api acme --provider Acme --format feed --source "$URL" ``` ``` Scaffolded acme (Acme, format feed): - docs/plugins.md plugins/feed/acme.go (new) plugins/feed/acme_test.go (new) ... @@ -71,7 +58,7 @@ go test ./plugins/feed -run TestAcme -update # writes testdata/golden/acme.jso **3. Check it, the way CI will.** ```sh -make plugins-json && make check +make check ``` ``` ... @@ -90,7 +77,7 @@ Most providers are an entry for a shared format plugin (`openapi`, `retirements` ## Supported plugins -**[docs/plugins.md](docs/plugins.md)** lists every plugin: provider, layer, what it watches, its source. The same list is on [docs.patch-it.dev](https://docs.patch-it.dev/supported-apis/plugins/), and as JSON in [docs/plugins.json](docs/plugins.json) (`make plugins-json`). +There's no list in this repo: each provider and source is registered in code with its metadata (`plugin.RegisterProvider`). `make plugins` prints the list as JSON, every release attaches it as `plugins.json`, and [docs.patch-it.dev](https://docs.patch-it.dev/supported-apis/plugins/) shows it. ## Layout @@ -98,7 +85,7 @@ Most providers are an entry for a shared format plugin (`openapi`, `retirements` |---|---| | [`core/plugin/`](core/plugin/) | The `Plugin` interface, the registry (`Register`, `Lookup`, `Names`), `Env`, `Result` | | [`plugins/`](plugins/) | The format plugins; `plugins/all` imports every one | -| [`cmd/new-plugin/`](cmd/new-plugin/), [`cmd/list-plugins/`](cmd/list-plugins/) | The starter for a new provider; the generator of `docs/plugins.json` | +| [`cmd/new-plugin/`](cmd/new-plugin/), [`cmd/list-plugins/`](cmd/list-plugins/) | The starter for a new provider; the list of providers and sources, printed from the registry | | [`core/kit/`](core/kit/), [`core/kit/kittest/`](core/kit/kittest/), [`core/github/`](core/github/) | What plugins share (the unchanged-page check, Wayback citing, the newest release or commit on GitHub) and the test kit (in-memory store, saved pages, golden files) | | [`core/event/`](core/event/), [`core/migrate/`](core/migrate/) | The change-event types of [patch-it's contract](https://github.com/patch-it-dev/patch-it/tree/main/docs/contract), and the v1 → v2 conversion | | [`core/diff/`](core/diff/), [`core/normalize/`](core/normalize/), [`core/hint/`](core/hint/), [`plugins/openapi/adapter/`](plugins/openapi/adapter/) | The OpenAPI diff engine: compare two specs, turn the changes into events, attach changelog hints; one adapter per spec provider | @@ -106,7 +93,7 @@ Most providers are an entry for a shared format plugin (`openapi`, `retirements` | [`contract/fixtures/`](contract/) | A copy of patch-it's change-event fixtures | | `plugin/`, `event/`, `fetch/`, `migrate/`, `kit/`, `kit/kittest/`, `retire/`, `sdkguide/` | Deprecated: the old import paths, forwarding to the packages above until code that imports them moves | | [`providers/`](providers/) | One page per provider: why that source, what the plugin reads | -| [`docs/`](docs/) | [The plugin list](docs/plugins.md), [how to write a plugin](docs/writing-a-plugin.md), the [README template](docs/README-template.md) patch-it's public repos share | +| [`docs/`](docs/) | [How to write a plugin](docs/writing-a-plugin.md), the [README template](docs/README-template.md) patch-it's public repos share | ## Run the checks diff --git a/cmd/list-plugins/main.go b/cmd/list-plugins/main.go index 55504d7..0718165 100644 --- a/cmd/list-plugins/main.go +++ b/cmd/list-plugins/main.go @@ -1,42 +1,34 @@ -// Command list-plugins writes docs/plugins.json: every registered plugin -// joined with its row in docs/plugins.md (id, provider, what it watches, -// format plugin, source URL, status, provider page), and sets the plugin -// count on the README's badge. It fails when the -// registry and the table disagree. Run it from the repo root: +// Command list-plugins prints what patch-it watches, as JSON: every registered +// provider with its sources (id, old slug, kind, layer, what it watches, the +// URLs it reads, its page), built from the plugin registry at run time. The +// Release workflow attaches its output to each release as plugins.json; it is +// never committed. // -// go run ./cmd/list-plugins # rewrite docs/plugins.json (make plugins-json) -// go run ./cmd/list-plugins -check # fail if docs/plugins.json is out of date (make check) -// go run ./cmd/list-plugins -o - # print it +// go run ./cmd/list-plugins # print it (make plugins) +// go run ./cmd/list-plugins -ref v0.3.0 -o f.json # link the pages at a tag, write a file package main import ( - "bytes" "flag" "fmt" "os" - "path/filepath" "github.com/patch-it-dev/patch-it-plugins/internal/pluginlist" _ "github.com/patch-it-dev/patch-it-plugins/plugins/all" ) func main() { - root := flag.String("root", ".", "the repo root") - out := flag.String("o", pluginlist.JSON, "where to write the list, relative to -root; - for stdout") - check := flag.Bool("check", false, "don't write: fail if the file differs from what would be written") + ref := flag.String("ref", "main", "the git ref the provider pages are linked at: a release tag, or main") + out := flag.String("o", "-", "where to write the list; - for stdout") flag.Parse() - if err := run(*root, *out, *check); err != nil { + if err := run(*ref, *out); err != nil { fmt.Fprintln(os.Stderr, "list-plugins:", err) os.Exit(1) } } -func run(root, out string, check bool) error { - plugins, err := pluginlist.List(root) - if err != nil { - return err - } - b, err := pluginlist.Marshal(plugins) +func run(ref, out string) error { + b, err := pluginlist.Marshal(pluginlist.Build(ref)) if err != nil { return err } @@ -44,24 +36,5 @@ func run(root, out string, check bool) error { _, err := os.Stdout.Write(b) return err } - readmePath := filepath.Join(root, pluginlist.Readme) - readme, err := os.ReadFile(readmePath) - if err != nil { - return err - } - badged, _ := pluginlist.Badge(readme, len(plugins)) - path := filepath.Join(root, out) - if check { - have, err := os.ReadFile(path) - if err != nil || !bytes.Equal(have, b) || !bytes.Equal(readme, badged) { - return fmt.Errorf("%s or the README's plugins badge is out of date: run `make plugins-json` and commit them", out) - } - return nil - } - if !bytes.Equal(readme, badged) { - if err := os.WriteFile(readmePath, badged, 0o644); err != nil { - return err - } - } - return os.WriteFile(path, b, 0o644) + return os.WriteFile(out, b, 0o644) } diff --git a/cmd/new-plugin/formats.go b/cmd/new-plugin/formats.go index 038f03f..673586b 100644 --- a/cmd/new-plugin/formats.go +++ b/cmd/new-plugin/formats.go @@ -4,7 +4,7 @@ package main // definition goes, the line that registers it, the placeholder sources its // test reads, and how the test runs it. type formatSpec struct { - layer string // default layer for docs/plugins.md + layer string // the source's default layer (plugin.Layers()) defDir string // the package directory the definition goes in def string // the definition file (template) register string // the plugin.Register line added to plugins/'s init (template) @@ -30,7 +30,7 @@ const todo = "TODO(new-plugin)" var formats = map[string]formatSpec{ "feed": { - layer: "API", + layer: "api", defDir: "plugins/feed", def: `package feed @@ -80,7 +80,7 @@ func {{.Lower}}Ingest(t *testing.T, file string) []event.EventV2 { }, "retirements": { - layer: "AI models", + layer: "ai-models", defDir: "plugins/retirements/retire", def: `package retire @@ -153,7 +153,7 @@ func {{.Lower}}Ingest(t *testing.T, file string) []event.EventV2 { }, "migrationguide": { - layer: "SDK", + layer: "sdk", defDir: "plugins/migrationguide/sdkguide", def: `package sdkguide @@ -331,7 +331,7 @@ func {{.Lower}}Ingest(t *testing.T, file string) []event.EventV2 { }, "apichangelog": { - layer: "API", + layer: "api", defDir: "plugins/apichangelog", def: `package apichangelog @@ -417,7 +417,7 @@ func {{.Lower}}Ingest(t *testing.T, file string) []event.EventV2 { }, "openapi": { - layer: "API", + layer: "api", defDir: "plugins/openapi/adapter", def: `package adapter diff --git a/cmd/new-plugin/main.go b/cmd/new-plugin/main.go index 43e94cb..12e21bc 100644 --- a/cmd/new-plugin/main.go +++ b/cmd/new-plugin/main.go @@ -7,14 +7,14 @@ // It writes the provider's definition in the format's package (or the reader // package the format uses: plugins/openapi/adapter/, plugins/retirements/retire/, // plugins/migrationguide/sdkguide/), adds its -// plugin.Register line to plugins/'s init, a test (registered, golden, -// drop) over placeholder sources with an empty golden file, a row in that -// testdata/README.md, providers/.md from providers/_template.md and a -// row in docs/plugins.md, then regenerates docs/plugins.json. `go test ./...` +// plugin.RegisterProvider block (the source's metadata) to plugins/'s +// init, a test (registered, golden, drop) over placeholder sources with an +// empty golden file, a row in that testdata/README.md, and providers/.md +// from providers/_template.md. `go test ./...` // passes right after; every place to fill in says TODO(new-plugin). // -// It changes nothing when the id is taken (registered, in docs/plugins.md, or -// a file it would write exists). +// It changes nothing when the id is taken (registered, or a file it would +// write exists). package main import ( @@ -37,7 +37,6 @@ import ( "unicode" "github.com/patch-it-dev/patch-it-plugins/core/plugin" - "github.com/patch-it-dev/patch-it-plugins/internal/pluginlist" _ "github.com/patch-it-dev/patch-it-plugins/plugins/all" ) @@ -47,13 +46,13 @@ type Options struct { API string // the plugin id: --api and every event's api Provider string // whose API, SDK or package Format string // the format plugin - Layer string // the docs/plugins.md layer; "" is the format's default + Layer string // the source's layer (plugin.Layers()); "" is the format's default Source string // the provider's source URL; "" is a placeholder - Watches string // the docs/plugins.md "what it watches" cell + Watches string // what the source watches, in a few words Page string // the provider page, providers/.md; "" is the id. An existing page is shared, not rewritten Today time.Time // Go runs the go command afterwards: the golden file from the placeholders - // (go test -update) and docs/plugins.json (go run ./cmd/list-plugins). + // (go test -update). Go bool } @@ -63,7 +62,7 @@ func main() { flag.StringVar(&o.API, "api", "", "the plugin id, e.g. acme or acme-models (lowercase letters, digits, hyphens)") flag.StringVar(&o.Provider, "provider", "", "the provider's name, e.g. Acme") flag.StringVar(&o.Format, "format", "", "the format plugin: "+strings.Join(formatNames(), ", ")) - flag.StringVar(&o.Layer, "layer", "", "the layer in docs/plugins.md: API, SDK, package, runtime, CI or \"AI models\" (default: the format's)") + flag.StringVar(&o.Layer, "layer", "", "the source's layer: api, sdk, package, runtime, ci, ai-models or infra (default: the format's)") flag.StringVar(&o.Source, "source", "", "the provider's source URL (default: a placeholder to fill in)") flag.StringVar(&o.Watches, "watches", "", "what it watches, in a few words (default: a placeholder)") flag.StringVar(&o.Page, "page", "", "the provider page, providers/.md (default: the --api id); one page per provider, so a second plugin of a provider names the existing page") @@ -116,8 +115,8 @@ func Run(o Options, out io.Writer) error { if o.Layer == "" { o.Layer = f.layer } - if _, ok := pluginlist.Layers[o.Layer]; !ok { - return fmt.Errorf("--layer %q: want API, SDK, package, runtime, CI or \"AI models\"", o.Layer) + if !slices.Contains(plugin.Layers(), plugin.Layer(o.Layer)) { + return fmt.Errorf("--layer %q: want one of %v", o.Layer, plugin.Layers()) } if o.Source == "" { o.Source = "https://example.com/TODO-" + o.API + "-source" @@ -143,20 +142,6 @@ func Run(o Options, out io.Writer) error { if _, ok := plugin.Lookup(o.API); ok { return fmt.Errorf("a plugin named %q is already registered: pick another --api", o.API) } - tablePath := filepath.Join(o.Root, pluginlist.Table) - table, err := os.ReadFile(tablePath) - if err != nil { - return fmt.Errorf("%v (run from the repo root, or pass --root)", err) - } - rows, err := pluginlist.ReadTable(table) - if err != nil { - return err - } - for _, r := range rows { - if r.ID == o.API { - return fmt.Errorf("%s already has a row for %q: pick another --api", pluginlist.Table, o.API) - } - } d := data{ID: o.API, Provider: o.Provider, Source: o.Source, Format: o.Format} d.Ident = ident(o.API) @@ -235,9 +220,6 @@ func Run(o Options, out io.Writer) error { if edits[readmePath], err = readmeRows(readme, []string{d.SourceFile, d.DropFile}, o.Source); err != nil { return fmt.Errorf("%s: %w", readmePath, err) } - if edits[pluginlist.Table], err = tableRow(table, o, d); err != nil { - return err - } // Write. var paths []string @@ -265,7 +247,6 @@ func Run(o Options, out io.Writer) error { if o.Go { for _, args := range [][]string{ {"test", "./" + pkgDir, "-run", "^Test" + d.Ident + "Golden$", "-update"}, - {"run", "./cmd/list-plugins"}, } { cmd := exec.Command("go", args...) cmd.Dir = o.Root @@ -273,7 +254,7 @@ func Run(o Options, out io.Writer) error { return fmt.Errorf("the files are written, but `go %s` failed: %w\n%s", strings.Join(args, " "), err, b) } } - fmt.Fprintf(out, " %s (from the placeholders)\n docs/plugins.json (regenerated)\n", filepath.Join(pkgDir, "testdata", "golden", o.API+".json")) + fmt.Fprintf(out, " %s (from the placeholders)\n", filepath.Join(pkgDir, "testdata", "golden", o.API+".json")) } fmt.Fprint(out, fill(nextSteps, struct { data @@ -293,8 +274,9 @@ Next steps (every place to fill in says TODO(new-plugin): grep -rn 'TODO(new-plu {{.Extra}}{{end}} 3. Write the golden file, then read it like code: go test ./plugins/{{.Format}} -run Test{{.Ident}} -update - 4. {{if .Shared}}Add {{.ID}} to providers/{{.Page}}.md (shared with the provider's other plugins){{else}}Fill in providers/{{.Page}}.md{{end}}, above all "Why this source", and the row in docs/plugins.md - (what it watches, the source; a plugin in the code is live), then: make plugins-json + 4. {{if .Shared}}Add {{.ID}} to providers/{{.Page}}.md (shared with the provider's other plugins){{else}}Fill in providers/{{.Page}}.md{{end}}, above all "Why this source", and the source's + metadata in its RegisterProvider block (what it watches, its URLs; a new provider's category and + homepage). go run ./cmd/list-plugins prints the list with it; there is no list to edit. 5. make check, then open the PR (git commit -s). ` @@ -401,7 +383,7 @@ func providerBlock(o Options, d data, line string) string { URLs: []string{%q}, Plugin: %s, }}, - })`, id, o.Provider, category, homepage, source, pluginlist.Layers[o.Layer], "providers/"+o.Page+".md", o.Watches, o.Source, expr) + })`, id, o.Provider, category, homepage, source, o.Layer, "providers/"+o.Page+".md", o.Watches, o.Source, expr) } // register adds line at the end of the init func that registers pkgDir's @@ -529,21 +511,6 @@ func readmeRows(readme []byte, files []string, source string) ([]byte, error) { return []byte(strings.Join(lines, "\n")), nil } -// tableRow adds the plugin's row at the end of docs/plugins.md's table. -func tableRow(table []byte, o Options, d data) ([]byte, error) { - end := bytes.Index(table, []byte(pluginlist.EndMarker)) - if end < 0 { - return nil, fmt.Errorf("%s: no %s", pluginlist.Table, pluginlist.EndMarker) - } - row := fmt.Sprintf("| [`%s`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/%s.md) | %s | %s | %s | `%s` | [%s](%s) | live |\n", - o.API, o.Page, o.Provider, o.Layer, o.Watches, o.Format, linkText(o.Source), o.Source) - var b bytes.Buffer - b.Write(table[:end]) - b.WriteString(row) - b.Write(table[end:]) - return b.Bytes(), nil -} - func linkText(url string) string { if strings.Contains(url, "example.com/TODO") { return "TODO: the source" diff --git a/cmd/new-plugin/main_test.go b/cmd/new-plugin/main_test.go index 209a5d4..3092ef5 100644 --- a/cmd/new-plugin/main_test.go +++ b/cmd/new-plugin/main_test.go @@ -17,7 +17,7 @@ import ( var today = time.Date(2026, 11, 19, 0, 0, 0, 0, time.UTC) // A plugin of every format, scaffolded into a copy of the repo: the copy -// builds, vets and passes its tests, and docs/plugins.json lists them all. +// builds, vets and passes its tests, and list-plugins lists them all. // Then the same command again changes nothing. func TestScaffoldEveryFormat(t *testing.T) { if testing.Short() { @@ -45,19 +45,21 @@ func TestScaffoldEveryFormat(t *testing.T) { t.Fatalf("go %s: %v\n%s", strings.Join(args, " "), err, b) } } - listed, err := os.ReadFile(filepath.Join(root, "docs", "plugins.json")) + list := exec.Command(goCmd, "run", "./cmd/list-plugins") + list.Dir = root + listed, err := list.Output() if err != nil { - t.Fatal(err) + t.Fatalf("go run ./cmd/list-plugins: %v", err) } for _, f := range formatNames() { - if !bytes.Contains(listed, []byte(`"id": "acme-`+f+`"`)) { - t.Errorf("docs/plugins.json doesn't list acme-%s", f) + if !bytes.Contains(listed, []byte(`"alias": "acme-`+f+`"`)) { + t.Errorf("list-plugins doesn't list acme-%s", f) } } before := tree(t, root) err = Run(Options{Root: root, API: "acme-feed", Provider: "Acme Corp", Format: "feed", Today: today, Go: true}, io.Discard) - if err == nil || !strings.Contains(err.Error(), "already has a row") { + if err == nil || !strings.Contains(err.Error(), "already exists") { t.Errorf("second run: err = %v, want the id taken", err) } if after := tree(t, root); !slices.Equal(before, after) { @@ -65,8 +67,8 @@ func TestScaffoldEveryFormat(t *testing.T) { } } -// A second plugin of a provider shares its page: no new page, and the row -// links the existing one. +// A second source of a provider shares its page and its provider: no new +// page, and the registration names the existing ones. func TestScaffoldSharesTheProviderPage(t *testing.T) { root := copyRepo(t) o := Options{Root: root, API: "algolia-analytics", Provider: "Algolia", Format: "openapi", Page: "algolia", Today: today} @@ -76,9 +78,11 @@ func TestScaffoldSharesTheProviderPage(t *testing.T) { if _, err := os.Stat(filepath.Join(root, "providers", "algolia-analytics.md")); err == nil { t.Error("wrote providers/algolia-analytics.md; want the shared providers/algolia.md") } - table, _ := os.ReadFile(filepath.Join(root, "docs", "plugins.md")) - if !bytes.Contains(table, []byte("[`algolia-analytics`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/algolia.md)")) { - t.Error("the row doesn't link providers/algolia.md") + reg, _ := os.ReadFile(filepath.Join(root, "plugins", "openapi", "openapi.go")) + for _, want := range []string{`ID: "algolia", Name: "Algolia"`, `ID: "analytics"`, `Page: "providers/algolia.md"`, `plugin.Category("search")`} { + if !bytes.Contains(reg, []byte(want)) { + t.Errorf("the registration has no %s", want) + } } } diff --git a/docs/README-template.md b/docs/README-template.md index 43f9bd5..e5fd3b8 100644 --- a/docs/README-template.md +++ b/docs/README-template.md @@ -23,7 +23,7 @@ In this order; drop one only if it can't be honest for the repo (no release yet: 3. **Language version**: e.g. `https://img.shields.io/github/go-mod/go-version/patch-it-dev/`. 4. **Latest release**, once there is one: `https://img.shields.io/github/v/release/patch-it-dev/`. 5. **License**: `https://img.shields.io/badge/license--3f5bf0`. -6. **A count the repo is about**, only if a test keeps it right (patch-it-plugins: the plugins badge, set by `make plugins-json` and checked by `go test`). +6. **No counts.** A badge that counts plugins, providers or anything else that changes at run time is a hand-kept list in disguise: link the docs instead. 7. **Docs**: `https://img.shields.io/badge/docs-docs.patch--it.dev-3f5bf0`. 8. **Discussions**: `https://img.shields.io/github/discussions/patch-it-dev/`. diff --git a/docs/README.md b/docs/README.md index fbfc72d..be3cdc9 100644 --- a/docs/README.md +++ b/docs/README.md @@ -2,10 +2,9 @@ | Page | What's in it | |---|---| -| [plugins.md](plugins.md) | Every supported plugin: id, provider, what it watches, source. Also published on [docs.patch-it.dev](https://docs.patch-it.dev/supported-apis/plugins/). | -| [plugins.json](plugins.json) | The same list as JSON, generated by `make plugins-json` from the plugin registry and plugins.md. Don't edit by hand. | +| [plugins.schema.json](plugins.schema.json) | The shape of `plugins.json`, the list of providers and sources that `go run ./cmd/list-plugins` prints from the registry and each release attaches. No list is committed. | | [README-template.md](README-template.md) | The shape every public patch-it repo's README follows: sections, badge set, words, images. | -| [writing-a-plugin.md](writing-a-plugin.md) | How to add a provider: the `Plugin` interface, fetching through `env.Get` and `kit/`, saving the source, the golden test with `kittest`, the provider page and the plugin list. | +| [writing-a-plugin.md](writing-a-plugin.md) | How to add a provider: the `Plugin` interface, fetching through `env.Get` and `kit/`, saving the source, the golden test with `kittest`, the provider page and the source's metadata in its registration. | | [../providers/](../providers/) | One page per provider: why that source, what the plugin reads and ignores. | | [../contract/README.md](../contract/README.md) | The change-event fixtures copied from patch-it. | diff --git a/docs/plugins.json b/docs/plugins.json deleted file mode 100644 index 48aadc0..0000000 --- a/docs/plugins.json +++ /dev/null @@ -1,307 +0,0 @@ -{ - "_comment": "Generated from the plugin registry and docs/plugins.md by `make plugins-json` (go run ./cmd/list-plugins). Don't edit by hand.", - "$schema": "./plugins.schema.json", - "count": 30, - "plugins": [ - { - "id": "algolia-ingestion", - "provider": "Algolia", - "layer": "api", - "watches": "Ingestion API operations deprecated, and deprecated operations removed, between releases of the spec", - "format": "openapi", - "source_url": "https://github.com/algolia/api-clients-automation", - "status": "live", - "provider_page": "providers/algolia.md" - }, - { - "id": "algolia-recommend", - "provider": "Algolia", - "layer": "api", - "watches": "Recommend API operations deprecated, and deprecated operations removed, between releases of the spec", - "format": "openapi", - "source_url": "https://github.com/algolia/api-clients-automation", - "status": "live", - "provider_page": "providers/algolia.md" - }, - { - "id": "algolia-search", - "provider": "Algolia", - "layer": "api", - "watches": "Search API operations deprecated, and deprecated operations removed, between releases of the spec", - "format": "openapi", - "source_url": "https://github.com/algolia/api-clients-automation", - "status": "live", - "provider_page": "providers/algolia.md" - }, - { - "id": "anthropic", - "provider": "Anthropic", - "layer": "sdk", - "watches": "Parameters removed from `messages.create()` and friends in the Python SDK 1.0", - "format": "migrationguide", - "source_url": "https://github.com/anthropics/anthropic-sdk-python/blob/v1.0.0/MIGRATION.md", - "status": "live", - "provider_page": "providers/anthropic.md" - }, - { - "id": "anthropic-models", - "provider": "Anthropic", - "layer": "ai-models", - "watches": "Model deprecations and retirement dates (Messages API models)", - "format": "retirements", - "source_url": "https://platform.claude.com/docs/en/about-claude/model-deprecations", - "status": "live", - "provider_page": "providers/anthropic-models.md" - }, - { - "id": "appwrite", - "provider": "Appwrite", - "layer": "api", - "watches": "REST API operations a server SDK calls that are deprecated between commits of the spec, SDK method renames left out", - "format": "openapi", - "source_url": "https://github.com/appwrite/specs", - "status": "live", - "provider_page": "providers/appwrite.md" - }, - { - "id": "cloudflare", - "provider": "Cloudflare", - "layer": "api", - "watches": "REST API operations Cloudflare deprecates, breaking on their end-of-life date when it gives one", - "format": "feed", - "source_url": "https://developers.cloudflare.com/fundamentals/api/reference/deprecations/index.xml", - "status": "live", - "provider_page": "providers/cloudflare.md" - }, - { - "id": "convex", - "provider": "Convex", - "layer": "api", - "watches": "Deployment API operations deprecated, and deprecated operations removed, between commits of the spec", - "format": "openapi", - "source_url": "https://github.com/get-convex/convex-backend", - "status": "live", - "provider_page": "providers/convex.md" - }, - { - "id": "datadog", - "provider": "Datadog", - "layer": "api", - "watches": "API v2 changes between OpenAPI spec releases", - "format": "openapi", - "source_url": "https://github.com/DataDog/datadog-api-client-python", - "status": "live", - "provider_page": "providers/datadog.md" - }, - { - "id": "elasticsearch", - "provider": "Elastic", - "layer": "api", - "watches": "Elasticsearch REST API operations deprecated, and deprecated operations removed, between released minor versions of the spec", - "format": "openapi", - "source_url": "https://github.com/elastic/elasticsearch-specification", - "status": "live", - "provider_page": "providers/elasticsearch.md" - }, - { - "id": "flagsmith", - "provider": "Flagsmith", - "layer": "api", - "watches": "REST API changes between releases of Flagsmith's own OpenAPI spec (removed endpoints and request parameters)", - "format": "openapi", - "source_url": "https://github.com/Flagsmith/flagsmith/blob/main/openapi.yaml", - "status": "live", - "provider_page": "providers/flagsmith.md" - }, - { - "id": "github", - "provider": "GitHub", - "layer": "api", - "watches": "REST API changes between dated API versions", - "format": "openapi", - "source_url": "https://github.com/github/rest-api-description", - "status": "live", - "provider_page": "providers/github.md" - }, - { - "id": "github-actions", - "provider": "GitHub", - "layer": "ci", - "watches": "GitHub Actions retirements, such as runner images and the `runs-on` labels they remove", - "format": "feed", - "source_url": "https://github.blog/changelog/label/actions/feed/", - "status": "live", - "provider_page": "providers/github-actions.md" - }, - { - "id": "gitlab", - "provider": "GitLab", - "layer": "api", - "watches": "REST API changes between releases, request side only (what callers send)", - "format": "openapi", - "source_url": "https://gitlab.com/gitlab-org/gitlab/-/tree/master/doc/api/openapi", - "status": "live", - "provider_page": "providers/gitlab.md" - }, - { - "id": "mongodb-atlas", - "provider": "MongoDB", - "layer": "api", - "watches": "Atlas Administration API versions deprecated and removed at their sunset date, per endpoint, from MongoDB's own API changelog", - "format": "apichangelog", - "source_url": "https://github.com/mongodb/openapi/blob/main/changelog/changelog.json", - "status": "live", - "provider_page": "providers/mongodb-atlas.md" - }, - { - "id": "neon", - "provider": "Neon", - "layer": "api", - "watches": "Neon API operations deprecated, and deprecated operations removed, between commits of the spec", - "format": "openapi", - "source_url": "https://github.com/neondatabase/neon-pkgs", - "status": "live", - "provider_page": "providers/neon.md" - }, - { - "id": "newrelic", - "provider": "New Relic", - "layer": "sdk", - "watches": "Members removed in the New Relic Python agent 11.0.0", - "format": "removedmembers", - "source_url": "https://github.com/newrelic/docs-website/blob/f0b4f817bf9a7b76a78e26609369a8a7ce764c8e/src/content/docs/release-notes/agent-release-notes/python-release-notes/python-agent-110000.mdx", - "status": "live", - "provider_page": "providers/newrelic.md" - }, - { - "id": "numpy", - "provider": "NumPy", - "layer": "package", - "watches": "Members removed or renamed in the main namespace in NumPy 2.0", - "format": "removedmembers", - "source_url": "https://github.com/numpy/numpy/blob/v2.0.0/doc/source/numpy_2_0_migration_guide.rst", - "status": "live", - "provider_page": "providers/numpy.md" - }, - { - "id": "openai-models", - "provider": "OpenAI", - "layer": "ai-models", - "watches": "Model deprecations and shutdown dates (Chat Completions and Responses text models)", - "format": "retirements", - "source_url": "https://developers.openai.com/api/docs/deprecations", - "status": "live", - "provider_page": "providers/openai-models.md" - }, - { - "id": "planetscale", - "provider": "PlanetScale", - "layer": "api", - "watches": "API operations deprecated, and deprecated operations removed, between commits of the spec, said in the operation's description", - "format": "openapi", - "source_url": "https://github.com/planetscale/terraform-provider-planetscale", - "status": "live", - "provider_page": "providers/planetscale.md" - }, - { - "id": "posthog", - "provider": "PostHog", - "layer": "sdk", - "watches": "Members removed in the posthog Python SDK 8.0.0", - "format": "removedmembers", - "source_url": "https://github.com/PostHog/posthog-python/blob/posthog-v8.0.0/docs/migration-7.x-to-8.0.md", - "status": "live", - "provider_page": "providers/posthog.md" - }, - { - "id": "redis", - "provider": "Redis", - "layer": "runtime", - "watches": "Commands removed entirely from Redis's own command catalog between two snapshots", - "format": "catalog", - "source_url": "https://github.com/redis/redis-doc/blob/master/commands.json", - "status": "live", - "provider_page": "providers/redis.md" - }, - { - "id": "salesforce", - "provider": "Salesforce", - "layer": "api", - "watches": "REST API version ranges deprecated and retired, from Salesforce's own API end-of-life policy page", - "format": "eol", - "source_url": "https://developer.salesforce.com/docs/platform/api-rest/guide/api-rest-eol.html", - "status": "live", - "provider_page": "providers/salesforce.md" - }, - { - "id": "sentry", - "provider": "Sentry", - "layer": "api", - "watches": "REST API operations deprecated between releases of the spec, most of them in the operation's description", - "format": "openapi", - "source_url": "https://github.com/getsentry/sentry/releases", - "status": "live", - "provider_page": "providers/sentry.md" - }, - { - "id": "stripe", - "provider": "Stripe", - "layer": "api", - "watches": "REST API changes between OpenAPI spec releases (removed and renamed endpoints, parameters and fields), with migration hints from Stripe's API changelog", - "format": "openapi", - "source_url": "https://github.com/stripe/openapi", - "status": "live", - "provider_page": "providers/stripe.md" - }, - { - "id": "supabase", - "provider": "Supabase", - "layer": "api", - "watches": "Management API operations deprecated, and deprecated operations removed, between commits of the spec", - "format": "openapi", - "source_url": "https://github.com/supabase/supabase", - "status": "live", - "provider_page": "providers/supabase.md" - }, - { - "id": "textual", - "provider": "Textual", - "layer": "package", - "watches": "Removals and breaking changes in release 0.86.0", - "format": "changelog", - "source_url": "https://github.com/Textualize/textual/blob/v0.86.0/CHANGELOG.md", - "status": "live", - "provider_page": "providers/textual.md" - }, - { - "id": "urllib3", - "provider": "urllib3", - "layer": "package", - "watches": "Removals and deprecations in release 2.0.0", - "format": "changelog", - "source_url": "https://github.com/urllib3/urllib3/blob/2.0.0/CHANGES.rst", - "status": "live", - "provider_page": "providers/urllib3.md" - }, - { - "id": "vercel", - "provider": "Vercel", - "layer": "api", - "watches": "REST API operations deprecated, and deprecated operations removed, between SDK releases of the spec", - "format": "openapi", - "source_url": "https://github.com/vercel/sdk", - "status": "live", - "provider_page": "providers/vercel.md" - }, - { - "id": "vertex-ai-models", - "provider": "Google", - "layer": "ai-models", - "watches": "Generative AI model deprecations on Vertex AI", - "format": "feed", - "source_url": "https://docs.cloud.google.com/feeds/generative-ai-on-vertex-ai-release-notes.xml", - "status": "live", - "provider_page": "providers/vertex-ai-models.md" - } - ] -} diff --git a/docs/plugins.md b/docs/plugins.md deleted file mode 100644 index 9d6f05f..0000000 --- a/docs/plugins.md +++ /dev/null @@ -1,56 +0,0 @@ -# Supported plugins - -Every source patch-it watches is one plugin. This is the list of them: **if a plugin isn't here, it doesn't exist yet**. Add a row when you add a plugin, and update it when you change one (see [AGENTS.md](../AGENTS.md)). - -Each plugin's id links to its provider page in [`providers/`](../providers/): why that source, what the plugin reads and ignores, and its known gaps. How to add one: [writing-a-plugin.md](writing-a-plugin.md). - -The table between the `plugins:start` and `plugins:end` markers is also published on [docs.patch-it.dev](https://docs.patch-it.dev/supported-apis/plugins/): the docs site reads this file from `main` when it builds. Keep it a plain Markdown table, one row per plugin, with absolute links only. [`plugins.json`](plugins.json) is the same list as JSON (id, provider, layer, format, source URL, status, provider page; schema in [`plugins.schema.json`](plugins.schema.json)), generated by `make plugins-json`. - - -| Plugin (`--api`) | Provider | Layer | What it watches | Format plugin | Source | Status | -|---|---|---|---|---|---|---| -| [`stripe`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/stripe.md) | Stripe | API | REST API changes between OpenAPI spec releases (removed and renamed endpoints, parameters and fields), with migration hints from Stripe's API changelog | `openapi` | [stripe/openapi](https://github.com/stripe/openapi) | live | -| [`datadog`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/datadog.md) | Datadog | API | API v2 changes between OpenAPI spec releases | `openapi` | [DataDog/datadog-api-client-python](https://github.com/DataDog/datadog-api-client-python) (`.generator/schemas/v2/openapi.yaml`) | live | -| [`github`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/github.md) | GitHub | API | REST API changes between dated API versions | `openapi` | [github/rest-api-description](https://github.com/github/rest-api-description) | live | -| [`gitlab`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/gitlab.md) | GitLab | API | REST API changes between releases, request side only (what callers send) | `openapi` | [gitlab-org/gitlab `doc/api/openapi`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/doc/api/openapi) | live | -| [`vercel`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/vercel.md) | Vercel | API | REST API operations deprecated, and deprecated operations removed, between SDK releases of the spec | `openapi` | [vercel/sdk](https://github.com/vercel/sdk) (`vercel-spec.json`) | live | -| [`supabase`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/supabase.md) | Supabase | API | Management API operations deprecated, and deprecated operations removed, between commits of the spec | `openapi` | [supabase/supabase](https://github.com/supabase/supabase) (`apps/docs/spec/api_v1_openapi.json`) | live | -| [`algolia-search`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/algolia.md) | Algolia | API | Search API operations deprecated, and deprecated operations removed, between releases of the spec | `openapi` | [algolia/api-clients-automation](https://github.com/algolia/api-clients-automation) (`specs/bundled/search.yml`) | live | -| [`algolia-ingestion`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/algolia.md) | Algolia | API | Ingestion API operations deprecated, and deprecated operations removed, between releases of the spec | `openapi` | [algolia/api-clients-automation](https://github.com/algolia/api-clients-automation) (`specs/bundled/ingestion.yml`) | live | -| [`algolia-recommend`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/algolia.md) | Algolia | API | Recommend API operations deprecated, and deprecated operations removed, between releases of the spec | `openapi` | [algolia/api-clients-automation](https://github.com/algolia/api-clients-automation) (`specs/bundled/recommend.yml`) | live | -| [`elasticsearch`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/elasticsearch.md) | Elastic | API | Elasticsearch REST API operations deprecated, and deprecated operations removed, between released minor versions of the spec | `openapi` | [elastic/elasticsearch-specification](https://github.com/elastic/elasticsearch-specification) (`output/openapi/elasticsearch-openapi.json`) | live | -| [`sentry`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/sentry.md) | Sentry | API | REST API operations deprecated between releases of the spec, most of them in the operation's description | `openapi` | [getsentry/sentry releases](https://github.com/getsentry/sentry/releases) (`openapi-derefed.json` on each `api@` release) | live | -| [`neon`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/neon.md) | Neon | API | Neon API operations deprecated, and deprecated operations removed, between commits of the spec | `openapi` | [neondatabase/neon-pkgs](https://github.com/neondatabase/neon-pkgs) (`packages/sdk/spec/neon-openapi.json`) | live | -| [`convex`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/convex.md) | Convex | API | Deployment API operations deprecated, and deprecated operations removed, between commits of the spec | `openapi` | [get-convex/convex-backend](https://github.com/get-convex/convex-backend) (`npm-packages/@convex-dev/platform/deployment-openapi.json`) | live | -| [`planetscale`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/planetscale.md) | PlanetScale | API | API operations deprecated, and deprecated operations removed, between commits of the spec, said in the operation's description | `openapi` | [planetscale/terraform-provider-planetscale](https://github.com/planetscale/terraform-provider-planetscale) (`schemas/openapi.yaml`) | live | -| [`mongodb-atlas`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/mongodb-atlas.md) | MongoDB | API | Atlas Administration API versions deprecated and removed at their sunset date, per endpoint, from MongoDB's own API changelog | `apichangelog` | [mongodb/openapi `changelog/changelog.json`](https://github.com/mongodb/openapi/blob/main/changelog/changelog.json) | live | -| [`openai-models`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/openai-models.md) | OpenAI | AI models | Model deprecations and shutdown dates (Chat Completions and Responses text models) | `retirements` | [OpenAI deprecations](https://developers.openai.com/api/docs/deprecations) | live | -| [`anthropic-models`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/anthropic-models.md) | Anthropic | AI models | Model deprecations and retirement dates (Messages API models) | `retirements` | [Anthropic model deprecations](https://platform.claude.com/docs/en/about-claude/model-deprecations) | live | -| [`anthropic`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/anthropic.md) | Anthropic | SDK | Parameters removed from `messages.create()` and friends in the Python SDK 1.0 | `migrationguide` | [anthropic-sdk-python MIGRATION.md](https://github.com/anthropics/anthropic-sdk-python/blob/v1.0.0/MIGRATION.md) | live | -| [`numpy`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/numpy.md) | NumPy | package | Members removed or renamed in the main namespace in NumPy 2.0 | `removedmembers` | [NumPy 2.0 migration guide](https://github.com/numpy/numpy/blob/v2.0.0/doc/source/numpy_2_0_migration_guide.rst) | live | -| [`textual`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/textual.md) | Textual | package | Removals and breaking changes in release 0.86.0 | `changelog` | [Textualize/textual CHANGELOG.md](https://github.com/Textualize/textual/blob/v0.86.0/CHANGELOG.md) | live | -| [`urllib3`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/urllib3.md) | urllib3 | package | Removals and deprecations in release 2.0.0 | `changelog` | [urllib3/urllib3 CHANGES.rst](https://github.com/urllib3/urllib3/blob/2.0.0/CHANGES.rst) | live | -| [`github-actions`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/github-actions.md) | GitHub | CI | GitHub Actions retirements, such as runner images and the `runs-on` labels they remove | `feed` | [GitHub changelog, Actions label (RSS)](https://github.blog/changelog/label/actions/feed/) | live | -| [`vertex-ai-models`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/vertex-ai-models.md) | Google | AI models | Generative AI model deprecations on Vertex AI | `feed` | [Vertex AI generative AI release notes (Atom)](https://docs.cloud.google.com/feeds/generative-ai-on-vertex-ai-release-notes.xml) | live | -| [`redis`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/redis.md) | Redis | runtime | Commands removed entirely from Redis's own command catalog between two snapshots | `catalog` | [redis/redis-doc `commands.json`](https://github.com/redis/redis-doc/blob/master/commands.json) | live | -| [`cloudflare`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/cloudflare.md) | Cloudflare | API | REST API operations Cloudflare deprecates, breaking on their end-of-life date when it gives one | `feed` | [Cloudflare API deprecations (RSS)](https://developers.cloudflare.com/fundamentals/api/reference/deprecations/index.xml) | live | -| [`posthog`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/posthog.md) | PostHog | SDK | Members removed in the posthog Python SDK 8.0.0 | `removedmembers` | [posthog-python 7.x to 8.0 migration guide](https://github.com/PostHog/posthog-python/blob/posthog-v8.0.0/docs/migration-7.x-to-8.0.md) | live | -| [`newrelic`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/newrelic.md) | New Relic | SDK | Members removed in the New Relic Python agent 11.0.0 | `removedmembers` | [Python agent 11.0.0 release notes](https://github.com/newrelic/docs-website/blob/f0b4f817bf9a7b76a78e26609369a8a7ce764c8e/src/content/docs/release-notes/agent-release-notes/python-release-notes/python-agent-110000.mdx) | live | -| [`appwrite`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/appwrite.md) | Appwrite | API | REST API operations a server SDK calls that are deprecated between commits of the spec, SDK method renames left out | `openapi` | [appwrite/specs](https://github.com/appwrite/specs) (`specs/.x/open-api3-.x.json`, the newest minor) | live | -| [`salesforce`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/salesforce.md) | Salesforce | API | REST API version ranges deprecated and retired, from Salesforce's own API end-of-life policy page | `eol` | [Salesforce API End-of-Life Policy](https://developer.salesforce.com/docs/platform/api-rest/guide/api-rest-eol.html) | live | -| [`flagsmith`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/flagsmith.md) | Flagsmith | API | REST API changes between releases of Flagsmith's own OpenAPI spec (removed endpoints and request parameters) | `openapi` | [github.com/Flagsmith/flagsmith/blob/main/openapi.yaml](https://github.com/Flagsmith/flagsmith/blob/main/openapi.yaml) | live | - - -## The columns - -| Column | Meaning | -|---|---| -| Plugin | The id you pass as `--api` and the `api` field on every event it emits. Lowercase, unique. | -| Provider | Whose API, SDK or package it is. | -| Layer | Which kind of dependency it covers: `API`, `SDK`, `package`, `runtime`, `CI` or `AI models`. | -| What it watches | The kind of change it reports, in a few words. | -| Format plugin | Which shared reader it is built on: `openapi` (a spec per release, diffed), `retirements` (a deprecations page), `migrationguide` (an SDK's migration guide), `removedmembers` (a package's table of removed members, in RST or Markdown), `changelog` (a release changelog file), `feed` (an RSS or Atom feed), `catalog` (a structured JSON catalog of names, diffed between two snapshots), `apichangelog` (a vendor's machine-readable API changelog, in JSON), `eol` (a vendor's own table of whole API version ranges and their support status). | -| Source | The provider's own public page or file the plugin reads. Every event links back to it. | -| Status | `live`: in the code on `main`. patch-it's hosted ingestion service runs it daily from the next release, which is cut automatically. `planned`: accepted, not written yet. `paused`: in the code but deliberately switched off, with the reason in the row. | - -Some plugins read one pinned release (`anthropic` 1.0.0, `newrelic` 11.0.0, `numpy` 2.0.0, `posthog` 8.0.0, `textual` 0.86.0, `urllib3` 2.0.0): a newer release of the same package is a new entry in that plugin, not a new row, unless it changes the format. diff --git a/docs/plugins.schema.json b/docs/plugins.schema.json index 35b1cad..2033d81 100644 --- a/docs/plugins.schema.json +++ b/docs/plugins.schema.json @@ -2,30 +2,49 @@ "$schema": "https://json-schema.org/draft/2020-12/schema", "$id": "https://github.com/patch-it-dev/patch-it-plugins/blob/main/docs/plugins.schema.json", "title": "patch-it plugins", - "description": "docs/plugins.json: every registered plugin, generated by `make plugins-json` from the plugin registry and docs/plugins.md.", + "description": "plugins.json: every registered provider and its sources, printed by `go run ./cmd/list-plugins` from the plugin registry and attached to each release. Never committed.", "type": "object", - "required": ["_comment", "$schema", "count", "plugins"], + "required": ["_comment", "$schema", "version", "provider_count", "source_count", "categories", "providers"], "additionalProperties": false, "properties": { "_comment": { "type": "string" }, "$schema": { "type": "string" }, - "count": { "type": "integer", "description": "How many plugins are listed." }, - "plugins": { + "version": { "type": "string", "minLength": 1, "description": "The release tag it was built from, or main." }, + "provider_count": { "type": "integer" }, + "source_count": { "type": "integer" }, + "categories": { + "type": "array", + "description": "Every category, in display order.", + "items": { "type": "string", "enum": ["ai", "payments", "infrastructure", "databases", "monitoring", "analytics", "auth", "messaging", "hosting", "search", "devtools", "crm", "packages"] } + }, + "providers": { "type": "array", "items": { "type": "object", - "required": ["id", "provider", "layer", "watches", "format", "source_url", "status", "provider_page"], + "required": ["id", "name", "category", "homepage", "sources"], "additionalProperties": false, "properties": { - "id": { "type": "string", "pattern": "^[a-z0-9][a-z0-9.-]*$", "description": "The plugin's --api id and every event's api field." }, - "provider": { "type": "string", "minLength": 1 }, - "layer": { "type": "string", "enum": ["api", "sdk", "package", "runtime", "ci", "ai-models"] }, - "watches": { "type": "string", "minLength": 1 }, - "format": { "type": "string", "enum": ["openapi", "retirements", "migrationguide", "removedmembers", "changelog", "feed", "apichangelog", "catalog", "eol"] }, - "source_url": { "type": "string", "pattern": "^https://" }, - "status": { "type": "string", "enum": ["live", "paused", "planned"] }, - "status_note": { "type": "string", "description": "Why, for a paused plugin." }, - "provider_page": { "type": "string", "pattern": "^providers/[a-z0-9][a-z0-9.-]*\\.md$", "description": "One page per provider: plugins of the same provider may share it." } + "id": { "type": "string", "pattern": "^[a-z0-9][a-z0-9-]*$" }, + "name": { "type": "string", "minLength": 1 }, + "category": { "type": "string", "enum": ["ai", "payments", "infrastructure", "databases", "monitoring", "analytics", "auth", "messaging", "hosting", "search", "devtools", "crm", "packages"] }, + "homepage": { "type": "string", "pattern": "^https://" }, + "sources": { + "type": "array", + "items": { + "type": "object", + "required": ["id", "alias", "kind", "layer", "watches", "urls", "page"], + "additionalProperties": false, + "properties": { + "id": { "type": "string", "pattern": "^[a-z0-9][a-z0-9-]*/[a-z0-9][a-z0-9-]*$", "description": "provider/source" }, + "alias": { "type": "string", "pattern": "^[a-z0-9][a-z0-9.-]*$", "description": "The old slug: --api and every event's api field until patch-it's change-event contract switches to source ids." }, + "kind": { "type": "string", "enum": ["openapi", "retirements", "migrationguide", "removedmembers", "changelog", "feed", "apichangelog", "catalog", "eol"] }, + "layer": { "type": "string", "enum": ["api", "sdk", "package", "runtime", "ci", "ai-models", "infra"] }, + "watches": { "type": "string", "minLength": 1 }, + "urls": { "type": "array", "items": { "type": "string", "pattern": "^https://" } }, + "page": { "type": "string", "pattern": "^https://github\\.com/patch-it-dev/patch-it-plugins/blob/[^/]+/providers/[a-z0-9][a-z0-9.-]*\\.md$" } + } + } + } } } } diff --git a/docs/writing-a-plugin.md b/docs/writing-a-plugin.md index cad93c2..4a19b9e 100644 --- a/docs/writing-a-plugin.md +++ b/docs/writing-a-plugin.md @@ -17,13 +17,12 @@ go run ./cmd/new-plugin --api acme --provider Acme --format openapi It writes, and lists: -- the provider's definition, in the package the format reads it from (`plugins/openapi/adapter/`, `plugins/retirements/retire/`, `plugins/migrationguide/sdkguide/` or `plugins//`), and its `plugin.Register` line in that format's `init` (plus the `core/migrate/` map entry that `retirements` and `migrationguide` need); +- the provider's definition, in the package the format reads it from (`plugins/openapi/adapter/`, `plugins/retirements/retire/`, `plugins/migrationguide/sdkguide/` or `plugins//`), and its `plugin.RegisterProvider` block (the provider and the source with their metadata) in that format's `init` (plus the `core/migrate/` map entry that `retirements` and `migrationguide` need); - `plugins//_test.go`: the registration test, a golden test and a pinned drop test, over placeholder sources in `testdata/`, with the golden file written from them; - a row per saved file in that `testdata/README.md` (URL, date, license, quote to fill in); - `providers/.md` from the template (or nothing, with `--page`, when the provider already has one); -- the row in `docs/plugins.md`, and `docs/plugins.json` regenerated. -`go test ./...` passes right after; every place to fill in says `TODO(new-plugin)` (`grep -rn 'TODO(new-plugin)' .`), and `make check` (its `todos` step, also in CI_go) fails until none is left. It refuses an id that's taken (registered, in `docs/plugins.md`, or a file it would write exists) and then changes nothing. The steps below are what you fill in. +`go test ./...` passes right after; every place to fill in says `TODO(new-plugin)` (`grep -rn 'TODO(new-plugin)' .`), and `make check` (its `todos` step, also in CI_go) fails until none is left. It refuses an id that's taken (registered, or a file it would write exists) and then changes nothing. The steps below are what you fill in. ## 2. Pick the source @@ -175,9 +174,9 @@ func TestGolden(t *testing.T) { Copy [`providers/_template.md`](../providers/_template.md) to `providers/.md` and fill in every section: what it watches; **why this source** (the link, the date you checked it, why it's the best place this provider announces breaking changes, the alternatives you didn't use and why, one short quote); how the provider announces changes; what the plugin reads and ignores; known gaps. -## 9. Add the row to `docs/plugins.md` +## 9. Fill in the metadata in the registration -**Not optional** ([AGENTS.md](../AGENTS.md) rule 1). Add one row between the `plugins:start` and `plugins:end` markers: the id linked to its provider page, the provider, what it watches, the format plugin, the source, the status. Absolute links only: the docs site copies the table as it is. Changing a plugin's id, source or what it watches updates its row in the same PR. Then run `make plugins-json` to regenerate `docs/plugins.json`; it fails, naming the problem, if the registry and the table disagree. +**Not optional** ([AGENTS.md](../AGENTS.md) rule 1). The `plugin.RegisterProvider` block the starter wrote is the only place the source is listed: the provider's id, name, category (`plugin.Categories()`) and homepage, and the source's id (`/`), layer (`plugin.Layers()`), what it watches, the URLs it reads and its page. A second source of a provider repeats the provider's id, name, category and homepage exactly. `make plugins` prints the list with it; each release attaches that list as `plugins.json`, which docs.patch-it.dev and patch-it's hosted ingestion service read. There's nothing else to update. ## 10. Check and open the PR diff --git a/internal/pluginlist/pluginlist.go b/internal/pluginlist/pluginlist.go index f798d86..e9a9aa8 100644 --- a/internal/pluginlist/pluginlist.go +++ b/internal/pluginlist/pluginlist.go @@ -1,253 +1,87 @@ -// Package pluginlist joins the plugin registry with the table in -// docs/plugins.md: the registry says which plugins exist and which format -// plugin each is built on, the table says who the provider is, what it -// watches, its source and its status. List checks that the two agree, so a -// plugin missing from the table (AGENTS.md rule 1) fails `make check`. +// Package pluginlist is the list of what patch-it watches, as JSON, built at +// run time from the plugin registry: every registered provider with its +// sources and their metadata. Nothing is kept by hand and nothing is +// committed: `go run ./cmd/list-plugins` prints it, and the Release workflow +// attaches it to each release as plugins.json (docs/plugins.schema.json +// describes it). package pluginlist import ( "bytes" "encoding/json" - "errors" - "fmt" - "os" - "path" - "path/filepath" - "reflect" - "regexp" - "slices" - "strconv" - "strings" "github.com/patch-it-dev/patch-it-plugins/core/plugin" ) -// Table is the plugin list, relative to the repo root. -const Table = "docs/plugins.md" +// Repo is where a source's page is published. +const Repo = "https://github.com/patch-it-dev/patch-it-plugins" -// JSON is the generated list, relative to the repo root. -const JSON = "docs/plugins.json" - -// The markers around the table in Table. -const ( - StartMarker = "" - EndMarker = "" -) - -// Row is one row of the table, as written. -type Row struct { - ID string // the plugin id, without the backticks and link - Page string // the provider page it links to: "algolia" for providers/algolia.md - Provider string - Layer string // as written: "API", "AI models" - Watches string - Format string - Source string // the cell, as written - Status string // the cell, as written: "live", "paused: why" -} - -// Plugin is one entry of docs/plugins.json. -type Plugin struct { - ID string `json:"id"` - Provider string `json:"provider"` - Layer string `json:"layer"` // api, sdk, package, runtime, ci or ai-models - Watches string `json:"watches"` - Format string `json:"format"` - SourceURL string `json:"source_url"` - Status string `json:"status"` // live, paused or planned - StatusNote string `json:"status_note,omitempty"` // why, for paused - ProviderPage string `json:"provider_page"` // relative to the repo root -} - -// File is docs/plugins.json. +// File is plugins.json. type File struct { - Comment string `json:"_comment"` - Schema string `json:"$schema"` - Count int `json:"count"` - Plugins []Plugin `json:"plugins"` -} - -var ( - cellID = regexp.MustCompile("^\\[`([a-z0-9][a-z0-9.-]*)`\\]\\(https://github\\.com/patch-it-dev/patch-it-plugins/blob/main/providers/([^)]+)\\.md\\)$") - cellFmt = regexp.MustCompile("^`([a-z]+)`$") - link = regexp.MustCompile(`\]\((https?://[^)\s]+)\)`) - statuses = []string{"live", "paused", "planned"} -) - -// Layers are the kinds of dependency a plugin covers: the table's word to -// the JSON's. -var Layers = map[string]string{ - "API": "api", - "SDK": "sdk", - "package": "package", - "runtime": "runtime", - "CI": "ci", - "AI models": "ai-models", -} - -// Statuses are the values of a plugin's status. -func Statuses() []string { return slices.Clone(statuses) } - -// ReadTable reads the rows between the markers of the plugin list. -func ReadTable(md []byte) ([]Row, error) { - start := bytes.Index(md, []byte(StartMarker)) - end := bytes.Index(md, []byte(EndMarker)) - if start < 0 || end < start { - return nil, fmt.Errorf("%s: no %s ... %s block", Table, StartMarker, EndMarker) - } - var rows []Row - lines := strings.Split(string(md[start+len(StartMarker):end]), "\n") - for _, l := range lines { - l = strings.TrimSpace(l) - if !strings.HasPrefix(l, "|") || strings.HasPrefix(l, "|---") || strings.HasPrefix(l, "| Plugin") { - continue - } - cells := splitRow(l) - if len(cells) != 7 { - return nil, fmt.Errorf("%s: row %q has %d cells, want 7", Table, l, len(cells)) - } - m := cellID.FindStringSubmatch(cells[0]) - if m == nil { - return nil, fmt.Errorf("%s: plugin cell %q is not [`id`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/.md)", Table, cells[0]) - } - f := cellFmt.FindStringSubmatch(cells[4]) - if f == nil { - return nil, fmt.Errorf("%s: %s: format cell %q is not `format`", Table, m[1], cells[4]) - } - rows = append(rows, Row{ID: m[1], Page: m[2], Provider: cells[1], Layer: cells[2], Watches: cells[3], Format: f[1], Source: cells[5], Status: cells[6]}) - } - return rows, nil + Comment string `json:"_comment"` + Schema string `json:"$schema"` + Version string `json:"version"` // the release it was built from; "main" between releases + Providers int `json:"provider_count"` + Sources int `json:"source_count"` + Categories []string `json:"categories"` // every category, in display order + List []Provider `json:"providers"` // sorted by id } -// splitRow splits "| a | b |" into its cells. A cell holding a pipe would -// have to escape it (\|); none does, and the docs site copies the table as is. -func splitRow(l string) []string { - l = strings.TrimSuffix(strings.TrimPrefix(l, "|"), "|") - var cells []string - for _, c := range strings.Split(l, "|") { - cells = append(cells, strings.TrimSpace(c)) - } - return cells +// Provider is one provider and its sources. +type Provider struct { + ID string `json:"id"` + Name string `json:"name"` + Category string `json:"category"` + Homepage string `json:"homepage"` + Sources []Source `json:"sources"` // sorted by id } -// Format is the format plugin a registered plugin is built on: the name of -// the package under plugins/ its type is defined in. -func Format(p plugin.Plugin) string { - t := reflect.TypeOf(p) - for t.Kind() == reflect.Pointer { - t = t.Elem() - } - return path.Base(t.PkgPath()) +// Source is one source. +type Source struct { + ID string `json:"id"` // provider/source + Alias string `json:"alias"` // the old slug: --api and every event's api until the contract switches to source ids + Kind string `json:"kind"` + Layer string `json:"layer"` + Watches string `json:"watches"` + URLs []string `json:"urls"` + Page string `json:"page"` // absolute, at Version } -// List joins the registered plugins with the table under root and checks -// they agree: every registered plugin has a row with its format, every row is -// registered (unless it's planned), and every plugin has a provider page. -// The plugins come out sorted by id. -func List(root string) ([]Plugin, error) { - md, err := os.ReadFile(filepath.Join(root, Table)) - if err != nil { - return nil, err +// Build is the list of everything registered, with pages linked at ref (a +// release tag, or "main"). +func Build(ref string) File { + if ref == "" { + ref = "main" } - rows, err := ReadTable(md) - if err != nil { - return nil, err + f := File{ + Comment: "Generated from the plugin registry by `go run ./cmd/list-plugins`. Not committed: each release carries it as an asset.", + Schema: Repo + "/blob/" + ref + "/docs/plugins.schema.json", + Version: ref, } - var errs []error - byID := map[string]Row{} - for _, r := range rows { - if _, dup := byID[r.ID]; dup { - errs = append(errs, fmt.Errorf("%s: %s has two rows", Table, r.ID)) - } - byID[r.ID] = r + for _, c := range plugin.Categories() { + f.Categories = append(f.Categories, string(c)) } - registered := map[string]string{} - for _, name := range plugin.Names() { - p, _ := plugin.Lookup(name) - registered[name] = Format(p) - if _, ok := byID[name]; !ok { - errs = append(errs, fmt.Errorf("%s is registered but has no row in %s: add one (AGENTS.md rule 1), then run `make plugins-json`", name, Table)) - } - } - var out []Plugin - for _, r := range rows { - status, note, _ := strings.Cut(r.Status, ":") - status, note = strings.TrimSpace(status), strings.TrimSpace(note) - if !slices.Contains(statuses, status) { - errs = append(errs, fmt.Errorf("%s: %s: status %q is not live, paused or planned", Table, r.ID, r.Status)) - } - layer, known := Layers[r.Layer] - if !known { - errs = append(errs, fmt.Errorf("%s: %s: layer %q is not one of API, SDK, package, runtime, CI, AI models", Table, r.ID, r.Layer)) - } - format, ok := registered[r.ID] - switch { - case !ok && status != "planned": - errs = append(errs, fmt.Errorf("%s: %s has a row but no plugin registers it: mark it planned, or remove the row", Table, r.ID)) - case ok && format != r.Format: - errs = append(errs, fmt.Errorf("%s: %s says format %q, but the plugin is in plugins/%s", Table, r.ID, r.Format, format)) + for _, p := range plugin.Providers() { + out := Provider{ID: p.ID, Name: p.Name, Category: string(p.Category), Homepage: p.Homepage} + for _, s := range p.Sources { + out.Sources = append(out.Sources, Source{ + ID: p.ID + "/" + s.ID, Alias: s.Plugin.Name(), Kind: s.Kind, Layer: string(s.Layer), + Watches: s.Watches, URLs: s.URLs, Page: Repo + "/blob/" + ref + "/" + s.Page, + }) } - // The row says what the source's registration says (moving to the - // registration only: until then, both). - if s, ok := plugin.LookupSource(r.ID); ok { - switch { - case s.Provider.Name != r.Provider: - errs = append(errs, fmt.Errorf("%s: %s: provider %q, but its registration says %q", Table, r.ID, r.Provider, s.Provider.Name)) - case string(s.Source.Layer) != layer: - errs = append(errs, fmt.Errorf("%s: %s: layer %q, but its registration says %q", Table, r.ID, layer, s.Source.Layer)) - case s.Source.Watches != r.Watches: - errs = append(errs, fmt.Errorf("%s: %s: what it watches differs from its registration's Watches", Table, r.ID)) - case "providers/"+r.Page+".md" != s.Source.Page: - errs = append(errs, fmt.Errorf("%s: %s: links providers/%s.md, but its registration's Page is %s", Table, r.ID, r.Page, s.Source.Page)) - } - } - // One page per provider: the algolia-* plugins share providers/algolia.md. - page := "providers/" + r.Page + ".md" - if _, err := os.Stat(filepath.Join(root, page)); err != nil { - errs = append(errs, fmt.Errorf("%s: %s links to %s, which doesn't exist (copy providers/_template.md)", Table, r.ID, page)) - } - src := link.FindStringSubmatch(r.Source) - if src == nil { - errs = append(errs, fmt.Errorf("%s: %s: the source cell has no absolute link", Table, r.ID)) - src = []string{"", ""} - } - out = append(out, Plugin{ - ID: r.ID, Provider: r.Provider, Layer: layer, Watches: r.Watches, Format: r.Format, - SourceURL: src[1], Status: status, StatusNote: note, ProviderPage: page, - }) - } - if len(errs) > 0 { - return nil, errors.Join(errs...) - } - slices.SortFunc(out, func(a, b Plugin) int { return strings.Compare(a.ID, b.ID) }) - return out, nil -} - -// Readme is the README, relative to the repo root: its plugins badge is a -// static count that `make plugins-json` keeps right. -const Readme = "README.md" - -var badge = regexp.MustCompile(`(img\.shields\.io/badge/plugins-)(\d+)(-)`) - -// Badge is readme with its plugins badge set to n. ok is false when it has none. -func Badge(readme []byte, n int) (out []byte, ok bool) { - if !badge.Match(readme) { - return readme, false + f.Sources += len(out.Sources) + f.List = append(f.List, out) } - return badge.ReplaceAll(readme, []byte("${1}"+strconv.Itoa(n)+"${3}")), true + f.Providers = len(f.List) + return f } -// Marshal is docs/plugins.json for plugins, ending in a newline. -func Marshal(plugins []Plugin) ([]byte, error) { +// Marshal is f as indented JSON, ending in a newline. +func Marshal(f File) ([]byte, error) { var buf bytes.Buffer enc := json.NewEncoder(&buf) enc.SetIndent("", " ") enc.SetEscapeHTML(false) - err := enc.Encode(File{ - Comment: "Generated from the plugin registry and docs/plugins.md by `make plugins-json` (go run ./cmd/list-plugins). Don't edit by hand.", - Schema: "./plugins.schema.json", - Count: len(plugins), - Plugins: plugins, - }) + err := enc.Encode(f) return buf.Bytes(), err } diff --git a/internal/pluginlist/pluginlist_test.go b/internal/pluginlist/pluginlist_test.go index 895d93f..21dfb84 100644 --- a/internal/pluginlist/pluginlist_test.go +++ b/internal/pluginlist/pluginlist_test.go @@ -1,109 +1,72 @@ package pluginlist import ( - "bytes" - "os" - "path/filepath" + "encoding/json" "strings" "testing" + "github.com/patch-it-dev/patch-it-plugins/core/plugin" _ "github.com/patch-it-dev/patch-it-plugins/plugins/all" ) const root = "../.." -// The real table agrees with the registry, and docs/plugins.json is what -// `make plugins-json` writes from them. This is how CI_go (go test) catches a -// plugin added without its row, or a row changed without regenerating. -func TestPluginsJSONIsUpToDate(t *testing.T) { - plugins, err := List(root) - if err != nil { - t.Fatal(err) - } - want, err := Marshal(plugins) - if err != nil { - t.Fatal(err) - } - have, err := os.ReadFile(filepath.Join(root, JSON)) - if err != nil { - t.Fatalf("%v (run `make plugins-json`)", err) - } - if !bytes.Equal(have, want) { - t.Errorf("%s is out of date: run `make plugins-json` and commit it", JSON) - } -} - -func TestReadTable(t *testing.T) { - md := "intro\n" + StartMarker + "\n" + - "| Plugin (`--api`) | Provider | Layer | What it watches | Format plugin | Source | Status |\n" + - "|---|---|---|---|---|---|---|\n" + - "| [`foo-api`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/foo-api.md) | Foo | API | Removed endpoints | `openapi` | [foo/spec](https://github.com/foo/spec) (`openapi.json`) | paused: not released yet |\n" + - EndMarker + "\n" - rows, err := ReadTable([]byte(md)) - if err != nil { - t.Fatal(err) - } - want := Row{ID: "foo-api", Page: "foo-api", Provider: "Foo", Layer: "API", Watches: "Removed endpoints", Format: "openapi", - Source: "[foo/spec](https://github.com/foo/spec) (`openapi.json`)", Status: "paused: not released yet"} - if len(rows) != 1 || rows[0] != want { - t.Errorf("rows = %+v, want [%+v]", rows, want) - } -} - -func TestReadTableRejectsABadRow(t *testing.T) { - for name, row := range map[string]string{ - "relative link": "| [`foo`](../providers/foo.md) | Foo | API | x | `openapi` | [s](https://s.example) | live |", - "not a provider page": "| [`foo`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/docs/foo.md) | Foo | API | x | `openapi` | [s](https://s.example) | live |", - "missing a cell": "| [`foo`](https://github.com/patch-it-dev/patch-it-plugins/blob/main/providers/foo.md) | Foo | `openapi` | [s](https://s.example) | live |", - } { - md := StartMarker + "\n" + row + "\n" + EndMarker - if _, err := ReadTable([]byte(md)); err == nil { - t.Errorf("%s: no error", name) +// The list is the registry: every registered source once, under its provider, +// with its old slug, and the counts agree. +func TestBuildListsEveryRegisteredSource(t *testing.T) { + f := Build("") + if f.Version != "main" { + t.Errorf("version %q, want main", f.Version) + } + seen := map[string]bool{} + n := 0 + for _, p := range f.List { + for _, s := range p.Sources { + n++ + if !strings.HasPrefix(s.ID, p.ID+"/") { + t.Errorf("%s is listed under %s", s.ID, p.ID) + } + if !strings.HasPrefix(s.Page, Repo+"/blob/main/providers/") { + t.Errorf("%s: page %s", s.ID, s.Page) + } + seen[s.Alias] = true } } -} - -// A registered plugin with no row fails the list, naming the rule. -func TestListWantsARowPerPlugin(t *testing.T) { - dir := t.TempDir() - if err := os.MkdirAll(filepath.Join(dir, "docs"), 0o755); err != nil { - t.Fatal(err) - } - if err := os.WriteFile(filepath.Join(dir, Table), []byte(StartMarker+"\n"+EndMarker+"\n"), 0o644); err != nil { - t.Fatal(err) + if n != f.Sources || len(f.List) != f.Providers || n != len(plugin.Sources()) { + t.Errorf("counts: %d sources listed, source_count %d, %d registered; %d providers, provider_count %d", n, f.Sources, len(plugin.Sources()), len(f.List), f.Providers) } - _, err := List(dir) - if err == nil || !strings.Contains(err.Error(), "stripe is registered but has no row") { - t.Errorf("err = %v, want stripe's missing row", err) + for _, name := range plugin.Names() { + if !seen[name] { + t.Errorf("registered plugin %s isn't listed", name) + } } } -// The README's plugins badge is a static number: it must be the count in -// docs/plugins.json (make plugins-json sets it), or the badge lies. -func TestReadmeBadgeCountsThePlugins(t *testing.T) { - plugins, err := List(root) +// One provider, several sources: Algolia's three APIs. +func TestAlgoliaIsOneProviderWithThreeSources(t *testing.T) { + b, err := Marshal(Build("v1.2.3")) if err != nil { t.Fatal(err) } - readme, err := os.ReadFile(filepath.Join(root, Readme)) - if err != nil { + var f File + if err := json.Unmarshal(b, &f); err != nil { t.Fatal(err) } - want, ok := Badge(readme, len(plugins)) - if !ok { - t.Fatal("README.md has no plugins badge (img.shields.io/badge/plugins--...)") - } - if !bytes.Equal(readme, want) { - t.Errorf("README.md's plugins badge doesn't say %d: run `make plugins-json`", len(plugins)) - } -} - -func TestBadge(t *testing.T) { - got, ok := Badge([]byte("[![Plugins](https://img.shields.io/badge/plugins-20-3f5bf0)](docs/plugins.md)"), 21) - if !ok || string(got) != "[![Plugins](https://img.shields.io/badge/plugins-21-3f5bf0)](docs/plugins.md)" { - t.Errorf("Badge = %s, %v", got, ok) - } - if _, ok := Badge([]byte("no badge"), 1); ok { - t.Error("found a badge in none") + for _, p := range f.List { + if p.ID != "algolia" { + continue + } + var ids []string + for _, s := range p.Sources { + ids = append(ids, s.ID+"="+s.Alias) + if !strings.Contains(s.Page, "/blob/v1.2.3/") { + t.Errorf("%s: page %s isn't at the release", s.ID, s.Page) + } + } + if got := strings.Join(ids, " "); got != "algolia/ingestion-api=algolia-ingestion algolia/recommend-api=algolia-recommend algolia/search-api=algolia-search" { + t.Errorf("algolia's sources: %s", got) + } + return } + t.Error("no algolia provider") } diff --git a/internal/pluginlist/schema_test.go b/internal/pluginlist/schema_test.go index 6cbc729..4b7df50 100644 --- a/internal/pluginlist/schema_test.go +++ b/internal/pluginlist/schema_test.go @@ -8,66 +8,95 @@ import ( "regexp" "slices" "testing" + + "github.com/patch-it-dev/patch-it-plugins/core/plugin" ) -// docs/plugins.json validates against docs/plugins.schema.json. The checker -// below knows only the keywords that schema uses (type, required, +// What list-plugins prints validates against docs/plugins.schema.json. The +// checker below knows only the keywords that schema uses (type, required, // properties, additionalProperties, items, enum, pattern, minLength): the // module takes no dependency for a JSON Schema library. -func TestPluginsJSONMatchesTheSchema(t *testing.T) { +func TestTheListMatchesTheSchema(t *testing.T) { var schema, doc any - for file, v := range map[string]*any{"docs/plugins.schema.json": &schema, JSON: &doc} { - b, err := os.ReadFile(filepath.Join(root, file)) + b, err := os.ReadFile(filepath.Join(root, "docs/plugins.schema.json")) + if err != nil { + t.Fatal(err) + } + if err := json.Unmarshal(b, &schema); err != nil { + t.Fatal(err) + } + for _, ref := range []string{"main", "v9.9.9"} { + out, err := Marshal(Build(ref)) if err != nil { t.Fatal(err) } - if err := json.Unmarshal(b, v); err != nil { - t.Fatalf("%s: %v", file, err) + if err := json.Unmarshal(out, &doc); err != nil { + t.Fatal(err) + } + for _, err := range validate(schema.(map[string]any), doc, "$") { + t.Error(err) } - } - for _, err := range validate(schema.(map[string]any), doc, "$") { - t.Error(err) } } -// The schema's enums are the values the generator writes. -func TestSchemaEnumsMatchTheGenerator(t *testing.T) { +// The schema's enums are the values the registry allows, and every kind. +func TestSchemaEnumsMatchTheRegistry(t *testing.T) { b, err := os.ReadFile(filepath.Join(root, "docs/plugins.schema.json")) if err != nil { t.Fatal(err) } var s struct { Properties struct { - Plugins struct { + Categories struct { Items struct { - Properties map[string]struct { - Enum []string `json:"enum"` + Enum []string `json:"enum"` + } `json:"items"` + } `json:"categories"` + Providers struct { + Items struct { + Properties struct { + Category struct { + Enum []string `json:"enum"` + } `json:"category"` + Sources struct { + Items struct { + Properties map[string]struct { + Enum []string `json:"enum"` + } `json:"properties"` + } `json:"items"` + } `json:"sources"` } `json:"properties"` } `json:"items"` - } `json:"plugins"` + } `json:"providers"` } `json:"properties"` } if err := json.Unmarshal(b, &s); err != nil { t.Fatal(err) } - props := s.Properties.Plugins.Items.Properties - var layers []string - for _, l := range Layers { - layers = append(layers, l) + var categories, layers []string + for _, c := range plugin.Categories() { + categories = append(categories, string(c)) + } + for _, l := range plugin.Layers() { + layers = append(layers, string(l)) } - for name, want := range map[string][]string{"layer": layers, "status": Statuses()} { - got := slices.Sorted(slices.Values(props[name].Enum)) - if !slices.Equal(got, slices.Sorted(slices.Values(want))) { - t.Errorf("schema enum for %s = %v, want %v", name, got, want) + src := s.Properties.Providers.Items.Properties.Sources.Items.Properties + for name, c := range map[string]struct{ got, want []string }{ + "categories": {s.Properties.Categories.Items.Enum, categories}, + "providers.category": {s.Properties.Providers.Items.Properties.Category.Enum, categories}, + "sources.layer": {src["layer"].Enum, layers}, + } { + if !slices.Equal(slices.Sorted(slices.Values(c.got)), slices.Sorted(slices.Values(c.want))) { + t.Errorf("schema enum %s = %v, want %v", name, c.got, c.want) } } - formats, err := filepath.Glob(filepath.Join(root, "plugins", "*")) + kinds, err := filepath.Glob(filepath.Join(root, "plugins", "*")) if err != nil { t.Fatal(err) } - for _, f := range formats { - if name := filepath.Base(f); name != "all" && !slices.Contains(props["format"].Enum, name) { - t.Errorf("format plugin %s is missing from the schema's format enum", name) + for _, k := range kinds { + if name := filepath.Base(k); name != "all" && !slices.Contains(src["kind"].Enum, name) { + t.Errorf("kind %s is missing from the schema's kind enum", name) } } } diff --git a/providers/README.md b/providers/README.md index d345a58..550b4fa 100644 --- a/providers/README.md +++ b/providers/README.md @@ -8,35 +8,4 @@ One page per plugin id: `providers/.md`. Start from [`_template.md`](_templ - **What the plugin reads** from it, and what it deliberately ignores (and why). - **Known gaps**: changes the provider makes that this plugin can't see yet. -| Page | Plugin | Format plugin | -|---|---|---| -| [stripe.md](stripe.md) | `stripe` | `openapi` | -| [datadog.md](datadog.md) | `datadog` | `openapi` | -| [github.md](github.md) | `github` | `openapi` | -| [gitlab.md](gitlab.md) | `gitlab` | `openapi` | -| [vercel.md](vercel.md) | `vercel` | `openapi` | -| [supabase.md](supabase.md) | `supabase` | `openapi` | -| [algolia.md](algolia.md) | `algolia-search`, `algolia-ingestion`, `algolia-recommend` | `openapi` | -| [elasticsearch.md](elasticsearch.md) | `elasticsearch` | `openapi` | -| [sentry.md](sentry.md) | `sentry` | `openapi` | -| [neon.md](neon.md) | `neon` | `openapi` | -| [convex.md](convex.md) | `convex` | `openapi` | -| [planetscale.md](planetscale.md) | `planetscale` | `openapi` | -| [appwrite.md](appwrite.md) | `appwrite` | `openapi` | -| [openai-models.md](openai-models.md) | `openai-models` | `retirements` | -| [anthropic-models.md](anthropic-models.md) | `anthropic-models` | `retirements` | -| [anthropic.md](anthropic.md) | `anthropic` | `migrationguide` | -| [numpy.md](numpy.md) | `numpy` | `removedmembers` | -| [posthog.md](posthog.md) | `posthog` | `removedmembers` | -| [newrelic.md](newrelic.md) | `newrelic` | `removedmembers` | -| [textual.md](textual.md) | `textual` | `changelog` | -| [urllib3.md](urllib3.md) | `urllib3` | `changelog` | -| [github-actions.md](github-actions.md) | `github-actions` | `feed` | -| [vertex-ai-models.md](vertex-ai-models.md) | `vertex-ai-models` | `feed` | -| [redis.md](redis.md) | `redis` | `catalog` | -| [cloudflare.md](cloudflare.md) | `cloudflare` | `feed` | -| [mongodb-atlas.md](mongodb-atlas.md) | `mongodb-atlas` | `apichangelog` | -| [salesforce.md](salesforce.md) | `salesforce` | `eol` | -| [flagsmith.md](flagsmith.md) | `flagsmith` | `openapi` | - -The list of plugins, with their status, is [docs/plugins.md](../docs/plugins.md). +Each source's registration names its page (`Page: "providers/.md"`); a provider's sources may share one. There's no list of pages here: `go run ./cmd/list-plugins` prints every provider and source with its page, and each release carries that list as `plugins.json`.