Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@

## Checklist

- [ ] **Metadata complete in the registration** (`plugin.RegisterProvider`: provider id, name, category, homepage; each source's id, layer, what it watches, URLs, page), and I read the words. There is no list to update. Not applicable: <!-- say why -->
- [ ] **Metadata complete in the registration** (`plugin.RegisterProvider`: provider id, name, category, homepage; each source's id, kind, layer, what it watches, URLs, packages, page), and I read the words. There is no list to update. Not applicable: <!-- say why -->
- [ ] Every commit is signed off (`git commit -s`, DCO)
- [ ] Golden test over a real saved copy of the source (`testdata/golden/<api>.json`)
- [ ] Golden test over a real saved copy of the source (`testdata/golden/<provider>-<source>.json`)
- [ ] `testdata/README.md` lists each file: URL, date fetched, license, quote of the part the plugin reads
- [ ] One pinned drop test: a realistic input that produces no event
- [ ] `providers/<api>.md` written or updated from `providers/_template.md`, including **Why this source**
- [ ] `providers/<provider>-<source>.md` written or updated from `providers/_template.md`, including **Why this source**
- [ ] Fetches only through `env.Get` / `core/kit/`
- [ ] `make check` passes locally (`make plugins` prints the source as you expect)
- [ ] Nothing private: no credentials, internal hosts, private links, customer names
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/CI_contract.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# contract/fixtures/ is a copy of patch-it's fixtures/ (the change-event contract
# patch-it owns). This fails when the two drift apart, so core/event/ and core/migrate/ test
# patch-it owns). This fails when the two drift apart, so core/event/ tests
# the same events the patch-it client does. Only top-level "_comment" fields may
# differ (scripts/check_contract.py).
#
Expand Down
12 changes: 4 additions & 8 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,10 @@ Refer to the service that runs these plugins as "patch-it's hosted ingestion ser

- **Start a new source with `make new-plugin ARGS='--provider <id> --name <Name> --source <id> --kind <kind>'`** (`go run ./cmd/new-plugin`; [which kind](docs/which-kind.md)), then fill in every `TODO(new-plugin)`; `make todos` (part of `make check` and CI_go) fails while one is left.
- **Fetch only through the environment** (`env.Get` or `core/kit/`). No `net/http` outside the fetch layer, no filesystem outside `testdata/`, no database.
- **Golden test + dated source quote**: a saved copy of the real source in `testdata/`, a row in `testdata/README.md` (URL, date fetched, license, quote), and `testdata/golden/<api>.json` with the exact events.
- **Golden test + dated source quote**: a saved copy of the real source in `testdata/`, a row in `testdata/README.md` (URL, date fetched, license, quote), and `testdata/golden/<provider>-<source>.json` with the exact events.
- **Precision over recall.** When unsure whether something is a change, don't emit an event.
- **One pinned drop test** per plugin: a realistic input that must produce no event.
- **`providers/<api>.md`** for every provider, copied from `providers/_template.md`, including **Why this source**: the link, the date it was checked, why it's the best source, the alternatives not used, and a short quote.
- **`providers/<provider>-<source>.md`** for every provider, copied from `providers/_template.md`, including **Why this source**: the link, the date it was checked, why it's the best source, the alternatives not used, and a short quote.
- **No source, no event**: every event carries the source URL and a quote.
- Don't edit golden files to make a test pass without explaining in the PR why the new output is right.

Expand Down Expand Up @@ -56,22 +56,18 @@ core/github/ the newest release / commit on GitHub, and a commit's SHA
core/versions/ every version of a source: GitHub releases, tags and commits, PyPI, npm, Web Archive captures
core/kit/ core/kit/kittest/ Unchanged, Cite, Wayback, Notices; the test kit (Mem, Pages, Ingest, Golden)
core/extract/ what readers of prose share: RSS/Atom entries, HTML as text, dates, code spans
core/event/ core/migrate/ the change-event types from patch-it's contract (v1 + v2), v1 -> v2
core/event/ the change event of patch-it's contract: types, fingerprint, validation
core/diff/ core/normalize/ core/hint/ the OpenAPI diff engine and its helpers
plugins/<kind>/ one package per kind, with its providers: openapi (+ adapter/, one spec adapter
per provider), notices (+ retire/, feed/, eol/), migrationguide (+ sdkguide/,
members/), changelog (Markdown, RST or JSON), catalog, apisurface (+ pysurface/);
plugins/all imports every one
plugins/removedmembers/ apichangelog/ retirements/ feed/ eol/ deprecated: merged into
migrationguide, changelog and notices, they only forward there
plugin/ event/ fetch/ migrate/ kit/ retire/ sdkguide/ deprecated old import paths: they only
forward to the packages above; don't add to them
cmd/list-plugins/ prints the list from the registry (make plugins); internal/pluginlist/ is its logic
cmd/new-plugin/ scaffolds a provider's source on a kind (make new-plugin ARGS=...; go run ./cmd/new-plugin --help)
contract/fixtures/ a copy of patch-it's change-event fixtures (CI_contract checks it)
Makefile make check (what CI_go runs, plus the leak guard), make plugins
docs/ contributor docs, including writing-a-plugin.md
providers/<api>.md one page per provider
providers/<provider>-<source>.md one page per provider
scripts/ leak_guard.py (hashed internal names), check_contract.py
.github/ CODEOWNERS, PR template, release-notes categories, workflows
(CI_go, CI_dco, CI_leak_guard, CI_contract, Release)
Expand Down
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@ Thanks for helping patch-it watch more APIs. This file is the rulebook; [docs/wr

1. **Give every provider and source complete metadata in its registration.** A source exists when its kind's package registers it with `plugin.RegisterProvider`: the provider's id, name, category and homepage, and the source's id, layer, what it watches, the URLs it reads and its page. That registration is the only list: `make plugins` prints it, each release carries it as `plugins.json`, and docs.patch-it.dev reads it from there. Nothing else to update.
2. **Fetch only through the environment.** Plugins get their sources from `env.Get` or the helpers in `core/kit/`, never from `net/http` directly, the filesystem or a database. That keeps tests offline and lets the hosted service cache, retry and rate-limit politely.
3. **Every plugin has a golden test** over a real saved copy of its source: `testdata/<file>` in, `testdata/golden/<api>.json` out. `testdata/README.md` lists each file with its URL, the date you fetched it, its license, and a short quote of the part your plugin reads. Keep openly licensed files whole; trim vendor documentation pages to what the test needs.
3. **Every plugin has a golden test** over a real saved copy of its source: `testdata/<file>` in, `testdata/golden/<provider>-<source>.json` out. `testdata/README.md` lists each file with its URL, the date you fetched it, its license, and a short quote of the part your plugin reads. Keep openly licensed files whole; trim vendor documentation pages to what the test needs.
4. **Precision over recall.** A missed change is a bug we can fix later. A wrong change becomes a wrong pull request in someone's repo. When in doubt, don't emit.
5. **One pinned drop test per plugin**: a realistic input your plugin must *not* turn into an event (an unrelated section, a renamed heading, a non-breaking addition).
6. **Document the provider** in `providers/<api>.md`, copied from [`providers/_template.md`](providers/_template.md): what it watches, **why you chose that source** (and the alternatives you didn't), how the provider announces changes, what the plugin reads and ignores, and known gaps.
6. **Document the provider** in `providers/<provider>-<source>.md`, copied from [`providers/_template.md`](providers/_template.md): what it watches, **why you chose that source** (and the alternatives you didn't), how the provider announces changes, what the plugin reads and ignores, and known gaps.
7. **Nothing private.** This repo is public. No credentials, tokens, internal hostnames or links to private repos or systems, in code, tests, commit messages or PR descriptions. CI scans every PR for secrets and for known internal names.

## Sign your commits (DCO)
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ test:

# A provider started with new-plugin is done when nothing it wrote still says
# TODO(new-plugin).
TODO_DIRS := plugins core/migrate providers
TODO_DIRS := plugins providers
todos:
@left=$$(grep -rl 'TODO(new-plugin)' $(TODO_DIRS) || true); \
if [ -n "$$left" ]; then \
Expand Down
2 changes: 1 addition & 1 deletion cmd/list-plugins/main.go
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
// Command list-plugins prints what patch-it watches, as JSON: every registered
// provider with its sources (id, old slug, kind, layer, what it watches, the
// provider with its sources (id, kind, layer, what it watches, the
// URLs it reads, its page), built from the plugin registry at run time. The
// Release workflow attaches its output to each release as plugins.json; it is
// never committed.
Expand Down
Loading
Loading