Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@

## Checklist

- [ ] **Added to / updated in `docs/plugins.md`** (new plugin: a row; changed id, provider, source, what it watches, format or status: the row updated; removed: the row removed). Not applicable: <!-- say why -->
- [ ] **Metadata complete in the registration** (`plugin.RegisterProvider`: provider id, name, category, homepage; each source's id, layer, what it watches, URLs, page), and I read the words. There is no list to update. Not applicable: <!-- say why -->
- [ ] Every commit is signed off (`git commit -s`, DCO)
- [ ] Golden test over a real saved copy of the source (`testdata/golden/<api>.json`)
- [ ] `testdata/README.md` lists each file: URL, date fetched, license, quote of the part the plugin reads
- [ ] One pinned drop test: a realistic input that produces no event
- [ ] `providers/<api>.md` written or updated from `providers/_template.md`, including **Why this source**
- [ ] Fetches only through `env.Get` / `kit/`
- [ ] `make check` passes locally (and `make plugins-json` was run if a plugin or its row changed)
- [ ] Fetches only through `env.Get` / `core/kit/`
- [ ] `make check` passes locally (`make plugins` prints the source as you expect)
- [ ] Nothing private: no credentials, internal hosts, private links, customer names

## Notes for the reviewer
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/CI_contract.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# contract/fixtures/ is a copy of patch-it's fixtures/ (the change-event contract
# patch-it owns). This fails when the two drift apart, so event/ and migrate/ test
# patch-it owns). This fails when the two drift apart, so core/event/ and core/migrate/ test
# the same events the patch-it client does. Only top-level "_comment" fields may
# differ (scripts/check_contract.py).
#
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/CI_go.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# The plugins build, are formatted and vetted, and their tests pass with the race
# detector. Two guards keep plugins self-contained:
# - imports: only this module, the standard library and the approved dependencies;
# - network: net/http only in the fetch layer (fetch/, kit/, adapter/).
# - network: net/http only in the fetch layer (core/fetch/, core/kit/, plugins/openapi/adapter/).
# The steps are Makefile targets, so `make check` runs exactly this locally.
# Until the Go code lands there is no go.mod, and every step after the first is skipped
# with a notice, so the check passes.
Expand Down Expand Up @@ -80,7 +80,6 @@ jobs:
run: make todos

# -race: the fetcher and the tests' HTTP servers run on other goroutines.
# Also fails when docs/plugins.json is out of date (make plugins-json).
- name: go test
if: steps.go.outputs.present == 'true'
run: make test
14 changes: 7 additions & 7 deletions .github/workflows/claude-code-review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -93,12 +93,12 @@ jobs:
any issue it references with `gh issue view <number>`. Read the diff with
`gh pr diff <PR NUMBER>`, and changed files in full where the diff isn't
enough context.
2. THE PLUGIN LIST (AGENTS.md Rule 1). Every plugin added, removed, or with a
changed id, provider, source, what it watches, format plugin or status has
its row added, removed or updated in docs/plugins.md, between the
`<!-- plugins:start -->` and `<!-- plugins:end -->` markers, as a plain
Markdown table with absolute links only. Compare the registered plugins with
the table yourself; a missing row is blocking.
2. THE METADATA (AGENTS.md Rule 1). Every source added or changed is
registered with plugin.RegisterProvider with complete, accurate metadata:
the provider's id, name, category and homepage; each source's id, layer,
what it watches, the URLs it reads and its page. There is no hand-kept list
(no docs/plugins.md, no plugins.json in the repo): one reappearing is
blocking.
3. PROVIDER PAGE. Every new provider has providers/<api>.md copied from
providers/_template.md, with a real **Why this source**: the link, the date
it was checked, why it's the best source, the alternatives not used, and a
Expand All @@ -110,7 +110,7 @@ jobs:
the source URL and a quote. A golden file edited without the PR explaining
why the new output is right is blocking.
5. PLUGIN CODE. Fetch only through env.Get or kit/: no net/http outside the
fetch layer (fetch/, kit/, adapter/), no filesystem outside testdata/, no
fetch layer (core/fetch/, core/kit/, plugins/openapi/adapter/), no filesystem outside testdata/, no
database. Precision over recall: flag any path that could emit an event for
something that isn't a real change; a wrong event becomes a wrong pull
request in someone's repo. Don't change contract/fixtures/ here first (the
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/claude.yml
Original file line number Diff line number Diff line change
Expand Up @@ -68,4 +68,4 @@ jobs:
--permission-mode default
--allowedTools "mcp__github_inline_comment__create_inline_comment,Read,Grep,Glob,Bash(gh pr view:*),Bash(gh pr diff:*),Bash(gh issue view:*)"
--disallowedTools "Edit,Write,MultiEdit,NotebookEdit,WebFetch,WebSearch,Bash(git push:*),Bash(git commit:*),Bash(git add:*),Bash(git rm:*),mcp__github_file_ops__commit_files,mcp__github_file_ops__delete_files"
--append-system-prompt "This is patch-it-plugins, a public repo of source plugins. Everything you write is public. You can only read and comment; you can't change files or run code, so if asked to, say so and describe the change instead. When asked to review a pull request (for example '@claude review'), check it against AGENTS.md: (1) every new, changed or removed plugin has its row added, updated or removed in docs/plugins.md between the plugins:start and plugins:end markers, absolute links only; (2) every new provider has providers/<api>.md from providers/_template.md with a real 'Why this source' (link, date checked, why it's best, alternatives, quote); (3) a golden test over a real saved source, a testdata/README.md row (URL, date fetched, license, quote), one pinned drop test, every event carries its source URL and a quote, and golden edits are explained; (4) fetch only through env.Get or kit/, no net/http outside fetch/, kit/ and adapter/, no filesystem outside testdata/; (5) precision over recall: flag anything that could emit an event for a non-change; (6) nothing private: no credentials, internal hostnames or service names, links to private repos or tickets, customer names, or links to AI sessions. You can't run the tests; say that CI_go covers them. Start with one verdict line ('✅ Ready to merge', '⚠️ Merge after fixes' or '❌ Not ready'), then a table of rule | status | evidence, blocking bugs, and clearly marked non-blocking suggestions. Be kind and specific: many authors are outside contributors. Never include a Claude session ID or claude.ai link."
--append-system-prompt "This is patch-it-plugins, a public repo of source plugins. Everything you write is public. You can only read and comment; you can't change files or run code, so if asked to, say so and describe the change instead. When asked to review a pull request (for example '@claude review'), check it against AGENTS.md: (1) every new or changed source is registered with plugin.RegisterProvider with complete, accurate metadata (provider id, name, category, homepage; source id, layer, what it watches, URLs, page), and no hand-kept list of plugins comes back; (2) every new provider has providers/<api>.md from providers/_template.md with a real 'Why this source' (link, date checked, why it's best, alternatives, quote); (3) a golden test over a real saved source, a testdata/README.md row (URL, date fetched, license, quote), one pinned drop test, every event carries its source URL and a quote, and golden edits are explained; (4) fetch only through env.Get or core/kit/, no net/http outside core/fetch/, core/kit/ and plugins/openapi/adapter/, no filesystem outside testdata/; (5) precision over recall: flag anything that could emit an event for a non-change; (6) nothing private: no credentials, internal hostnames or service names, links to private repos or tickets, customer names, or links to AI sessions. You can't run the tests; say that CI_go covers them. Start with one verdict line ('✅ Ready to merge', '⚠️ Merge after fixes' or '❌ Not ready'), then a table of rule | status | evidence, blocking bugs, and clearly marked non-blocking suggestions. Be kind and specific: many authors are outside contributors. Never include a Claude session ID or claude.ai link."
11 changes: 10 additions & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -172,9 +172,18 @@ jobs:
git tag -a "$TAG" "$SHA" -m "$TAG" -m "Released by the Release workflow ($by)."
git push origin "refs/tags/$TAG"

# The list of providers and sources, from the registry at the tag: the
# release's plugins.json asset. Never committed; the docs site reads the
# latest release's copy.
- name: list the providers and sources
if: steps.version.outputs.skip != 'true'
env:
TAG: ${{ steps.version.outputs.tag }}
run: go run ./cmd/list-plugins -ref "$TAG" -o "$RUNNER_TEMP/plugins.json"

- name: publish the GitHub release
if: steps.version.outputs.skip != 'true'
env:
GH_TOKEN: ${{ github.token }}
TAG: ${{ steps.version.outputs.tag }}
run: gh release create "$TAG" --verify-tag --generate-notes --title "$TAG"
run: gh release create "$TAG" --verify-tag --generate-notes --title "$TAG" "$RUNNER_TEMP/plugins.json"
40 changes: 21 additions & 19 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,11 @@

Rules for AI coding agents (and humans) working in this repo. `CLAUDE.md` imports this file.

## Rule 1: every new or changed plugin goes in `docs/plugins.md`
## Rule 1: every new or changed provider or source has complete metadata in its registration

**If you add a plugin, add its row to [`docs/plugins.md`](docs/plugins.md) in the same PR. If you change a plugin's id, provider, source, what it watches, format plugin or status, update its row in the same PR. If you remove one, remove its row.**
**A source exists when its kind's package registers it with `plugin.RegisterProvider`, and its metadata lives only there: the provider's id, name, category and homepage, and each source's id, layer, what it watches, the URLs it reads and its page, `providers/<name>.md`.** Adding, changing or removing a source is a change to that registration (and to its page) in the same PR. There is no list to update: nothing in this repo, the docs site or anywhere else keeps one by hand.

The list of supported APIs on docs.patch-it.dev is generated from that file, so a plugin missing from it doesn't exist as far as users can tell. Then run `make plugins-json`: it regenerates [`docs/plugins.json`](docs/plugins.json) from the plugin registry and the table (and the plugin count on the README's badge), and fails if a registered plugin has no row (or a row has no plugin, a wrong format or no provider page). `go test` fails while `docs/plugins.json` is out of date. The check can't tell whether a row's words are right: you are that check. Say in the PR that you compared them.

Keep the table between the `<!-- plugins:start -->` and `<!-- plugins:end -->` markers a plain Markdown table, one row per plugin, absolute links only (the docs site copies it verbatim).
`go run ./cmd/list-plugins` prints the list from the registry (JSON), and the Release workflow attaches it to every release as `plugins.json`; the docs site and patch-it's hosted ingestion service read it from there. `RegisterProvider` refuses incomplete metadata and `go test` fails when a source's page is missing, but no check can tell whether the words are right: you are that check. Say in the PR that you read them.

## Rule 2: this repo is public

Expand All @@ -26,7 +24,7 @@ Refer to the service that runs these plugins as "patch-it's hosted ingestion ser
## Rules for plugin code

- **Start a new provider with `go run ./cmd/new-plugin --api <id> --provider <Name> --format <format>`**, then fill in every `TODO(new-plugin)`; `make todos` (part of `make check` and CI_go) fails while one is left.
- **Fetch only through the environment** (`env.Get` or `kit/`). No `net/http` outside the fetch layer, no filesystem outside `testdata/`, no database.
- **Fetch only through the environment** (`env.Get` or `core/kit/`). No `net/http` outside the fetch layer, no filesystem outside `testdata/`, no database.
- **Golden test + dated source quote**: a saved copy of the real source in `testdata/`, a row in `testdata/README.md` (URL, date fetched, license, quote), and `testdata/golden/<api>.json` with the exact events.
- **Precision over recall.** When unsure whether something is a change, don't emit an event.
- **One pinned drop test** per plugin: a realistic input that must produce no event.
Expand All @@ -41,7 +39,7 @@ Refer to the service that runs these plugins as "patch-it's hosted ingestion ser
```sh
make check # tidy, build, gofmt, vet, the import allowlist, the net/http guard, go test -race, no TODO(new-plugin) left, leak guard
make check PATCH_IT=../patch-it # also CI_contract: contract/fixtures/ against a patch-it checkout
make plugins-json # regenerate docs/plugins.json after adding or changing a plugin or its row
make plugins # print the list of providers and sources from the registry (what each release attaches as plugins.json)
```

One target at a time: `make tidy`, `make build`, `make fmt`, `make vet`, `make imports`, `make nethttp`, `make test`, `make todos`, `make leak-guard`, `make contract`. Use `make GO=/path/to/go check` for another Go.
Expand All @@ -51,19 +49,23 @@ One target at a time: `make tidy`, `make build`, `make fmt`, `make vet`, `make i
Go module `github.com/patch-it-dev/patch-it-plugins`; the only dependency is `gopkg.in/yaml.v3`.

```
plugin/ the Plugin interface, the registry, Env, Result, Spec (baselines)
cmd/list-plugins/ writes docs/plugins.json (make plugins-json); internal/pluginlist/ is its logic
core/plugin/ the Plugin interface (Run; ListVersions, ReadAt, Extract), the registry of providers and sources, Env, Result
core/fetch/ the HTTP fetch with retries and the ETag / Last-Modified cache
core/github/ the newest release / commit on GitHub, and a commit's SHA and date (one place)
core/versions/ every version of a source: GitHub releases, tags and commits, PyPI, npm, the Web Archive at a date
core/kit/ core/kit/kittest/ Unchanged, Cite, Wayback; the test kit (Mem, Pages, Ingest, Golden)
core/event/ core/migrate/ the change-event types from patch-it's contract (v1 + v2), v1 -> v2
core/diff/ core/normalize/ core/hint/ the OpenAPI diff engine and its helpers
plugins/<kind>/ one package per kind, with its providers: openapi (+ adapter/, one spec adapter
per provider), retirements (+ retire/), migrationguide (+ sdkguide/),
removedmembers, changelog, feed, apichangelog, catalog, eol;
plugins/all imports every one
plugin/ event/ fetch/ migrate/ kit/ retire/ sdkguide/ deprecated old import paths: they only
forward to the packages above; don't add to them
cmd/list-plugins/ prints the list from the registry (make plugins); internal/pluginlist/ is its logic
cmd/new-plugin/ scaffolds a provider on a format plugin (go run ./cmd/new-plugin --help)
plugins/<format>/ one package per format plugin: openapi, retirements, migrationguide,
removedmembers, changelog, feed, apichangelog, catalog; plugins/all imports every one
kit/ kit/kittest/ Unchanged, Cite, Wayback; the test kit (Mem, Pages, Ingest, Golden)
event/ migrate/ the change-event types from patch-it's contract (v1 + v2), v1 -> v2
fetch/ the HTTP fetch with retries and the ETag / Last-Modified cache
diff/ normalize/ hint/ adapter/ the OpenAPI diff engine and the spec adapters
retire/ sdkguide/ deprecations-page and migration-guide readers
contract/fixtures/ a copy of patch-it's change-event fixtures (CI_contract checks it)
docs/plugins.md the list of supported plugins (Rule 1); docs/plugins.json is generated from it
Makefile make check (what CI_go runs, plus the leak guard), make plugins-json
Makefile make check (what CI_go runs, plus the leak guard), make plugins
docs/ contributor docs, including writing-a-plugin.md
providers/<api>.md one page per provider
scripts/ leak_guard.py (hashed internal names), check_contract.py
Expand All @@ -77,7 +79,7 @@ The [change-event contract](https://github.com/patch-it-dev/patch-it/tree/main/d

- Sign off every commit: `git commit -s` (DCO; `CI_dco` checks it).
- Subjects short, lowercase, imperative: `add the foo plugin`.
- One plugin or fix per PR; fill in every checkbox of the PR template, including "added to docs/plugins.md".
- One plugin or fix per PR; fill in every checkbox of the PR template, including "metadata complete in the registration".
- AI-assisted commits keep their co-author trailer.
- Never re-run, dispatch or retrigger CI to "see if it passes"; fix the cause and push.

Expand Down
Loading
Loading