Skip to content

add the salesforce eol plugin - #137

Merged
adssib merged 4 commits into
mainfrom
salesforce-source
Nov 20, 2026
Merged

adssib merged 4 commits into
mainfrom
salesforce-source

Conversation

@AbdullahT01

Copy link
Copy Markdown
Contributor

What this changes

Adds Salesforce as a new plugin, on a new format plugin, plugins/eol. Closes #101.

Salesforce's REST API has no stable spec, no changelog feed, and no official Python or JS SDK (researched and ruled out below). The one real, authoritative, dated source is Salesforce's own API End-of-Life Policy page: a table of whole numbered API version ranges (/services/data/vNN.N/), each marked supported, already retired, or scheduled to be. No existing format plugin fits it:

Why not an existing format Reason
openapi No stable, versioned spec exists to diff (only a live, per-org beta generator for sObjects only, explicitly not for production use, nothing pinned to a release).
retirements Built for a named model per row with a replacement column (markdown table). Salesforce's rows are version ranges with no replacement (moving to any newer version clears it), in an HTML table, not markdown.
feed / apichangelog / changelog / catalog / removedmembers / migrationguide No feed, no machine-readable changelog, no package, no catalog file.

eol reuses the same shape as retirements/feed (read the live page now, cite a Wayback Machine snapshot, no diffing between two states needed since the page itself already lists every past and scheduled retirement with dates) but with its own HTML-table parser and Subject.Type = api_operation instead of model, since this isn't about a named model.

The Subject/Operation choice, flagged for review: a whole version's retirement doesn't name one specific operation. I didn't invent a wildcard operation string (no contract change, no new Subject type): each event cites GET /services/data/vNN.N/, Salesforce's own real, documented "List Available REST Resources" call, which is the one honest, literal resource that exists at every version and nothing else. The description and provider page Known gaps are upfront that in practice every resource under that version is gone, not just this one.

Dates: Salesforce gives seasons ("Summer '27"), not calendar days, and its own production rollout is staggered by instance with no single verifiable date per season (checked against four independent sources, which disagree on the exact weekend). Rather than guess a date, the event's version is the API version number and the season is quoted verbatim in the description and excerpt.

Golden test doubles as the historical replay, same as retirements: the single real, saved snapshot already contains two genuine past retirements (versions 7.0–20.0 retired Summer '22, versions 21.0–30.0 retired Summer '25) plus one real scheduled one (versions 31.0–40.0, deprecated from Summer '27), 34 events total; the still-"Supported." 41.0–67.0 band is the pinned drop test.

Checklist

  • Added to docs/plugins.md (new row) and make plugins-json run (also added eol to docs/plugins.schema.json's format enum, and to docs/writing-a-plugin.md's format-plugin table — not yet wired into cmd/new-plugin --format, noted there)
  • Every commit is signed off (git commit -s, DCO)
  • Golden test over a real saved copy of the source (testdata/golden/salesforce.json)
  • testdata/README.md lists the file: URL, date fetched, license, quote
  • One pinned drop test: the real "Supported." band, no event
  • providers/salesforce.md written from providers/_template.md, including Why this source
  • Fetches only through env.Get (via kit.Cite/kit.Wayback, same as retirements and feed)
  • make check passes locally
  • Nothing private: no credentials, internal hosts, private links, customer names

Notes for the reviewer

  • This is a new format plugin (plugins/eol), not a new provider on an existing one — please look at the Subject/Operation design choice above in particular.
  • cmd/new-plugin --format doesn't scaffold eol yet (only the original 8 formats); a future provider on this format has to be hand-written, same as this one. Left out of scope here.

@AbdullahT01
AbdullahT01 requested a review from adssib as a code owner November 20, 2026 03:44
@claude

claude Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

Claude finished @AbdullahT01's task in 1m 36s —— View job


⚠️ Merge after fixes. The code, tests and docs are in good shape, but the provider page says the live Salesforce page returns 403 to a plain fetch, and Run depends on fetching that live page. Please confirm the fetch works in production (details below).

Thanks for this, @AbdullahT01. The write-up of why no existing format fits, and of why the season names aren't turned into dates, made this easy to review. 🙏

Rules

Rule Status Evidence
Plugin list (Rule 1) ✅ docs/plugins.md:40 has a new salesforce row: absolute links, format eol, status live. docs/plugins.json and the README badge (28 → 29) were regenerated. go test passes, so the JSON is up to date. I compared the row's wording with the plugin and the provider page, and they match.
Provider page ⚠️ providers/salesforce.md follows the template. Why this source has the link, the date checked (2026-11-20), why it's the best source, the alternatives (changelog/RSS, OpenAPI, SDKs, the detail sub-pages) and a quote. One sentence conflicts with the code: see Bugs.
Golden test + quote ✅ testdata/golden/salesforce.json holds 34 events. testdata/README.md has the URL, the fetch date, the license and the quote, and says how the file was trimmed.
Drop test ✅ TestDropsAStillSupportedBand checks that the real "Versions 41.0 through 67.0: Supported." row produces no event.
Every event cites a source ✅ SourceURL is the Wayback snapshot and SourceExcerpt is the row's label plus its status text.
Fetch only via env ✅ Fetches go through env.Get and kit.Cite/kit.Wayback. No net/http, no filesystem, no database. Same pattern as retirements.
Nothing private (Rule 2) ✅ I found nothing private in the code, docs, test data, commit message or PR body. The leak guard passes. The commit is signed off.

Bugs

  1. The plugin may fail on every run in production (plugins/eol/eol.go:171, providers/salesforce.md:11, inline comments posted).
    • The conflict: the provider page says "The live page 403s a plain fetch (bot-protected), so it's read through a Wayback Machine snapshot." But Run first calls env.Get(ctx, pg.URL) on the live page. Wayback is only used afterwards, to cite the page, never in its place. If that GET gets a 403, Run returns an error every time and no event is ever emitted.
    • Why the tests don't catch it: the golden test uses a hand-trimmed copy of the page, so it can't show whether the HTML the service actually receives contains the <tbody>/<tr>/<td> table that Parse matches. developer.salesforce.com/docs/... pages may load their content in the browser.
    • The same doubt for the snapshot: kit.Cite re-parses the raw archived page (the id_ snapshot), so that page also has to parse the same way.
    • What I need: please confirm both the live fetch and the raw snapshot parse correctly, and reword the provider page; or choose a source the plugin can actually fetch. My sandbox couldn't make network requests, so I couldn't check this myself.

No other correctness problems found. I checked these:

  • The order the status regexes are tried in: a "Supported. Deprecated… Retired…" row becomes a deprecation event, as intended.
  • A version range expands to one event per version.
  • ChangeType is part of the fingerprint, so when 31–40 later move from deprecated to retired, they produce new breaking events instead of being deduplicated against the old ones.
  • A page with no table or no version rows returns an error instead of emitting nothing.

On the Subject design you asked about: using GET /services/data/vNN.N/ with api_operation seems reasonable to me, because it is a real, documented call and the Known gaps section says plainly what it stands for.

Verified

Run on this branch (e528a45):

  • go build ./... → ok
  • go vet ./... → ok
  • gofmt -l . → printed nothing
  • go test -race ./... → every package ok, including plugins/eol and internal/pluginlist
  • python3 scripts/leak_guard.py --all → "no banned terms (8 hashed terms checked)"
  • Not run: make imports, make nethttp and make todos needed approval in this sandbox, and I couldn't fetch the live URL.

Non-blocking (optional)

  • Excerpt wording: the excerpt's label is rebuilt as "Versions %d.0 through %d.0" rather than copied from the cell. That matches today's page, but if Salesforce ever writes "Versions 7 through 20", the quote would no longer be word for word. You could use text(cells[0][1]) instead.
  • Version regex: versions drops anything after the decimal point. That's fine for the current whole-number ranges, but worth a comment so nobody is surprised later.
  • <td> with attributes: the cell regex only matches a bare <td>. If the page adds a class to its cells, every row would be skipped and Parse would return its "shape may have changed" error. That's safe, since no wrong events come out, but <td[^>]*> would make it more robust.

Comment thread plugins/eol/eol.go Outdated
// one GET and no snapshot.
func (p Plugin) Run(ctx context.Context, env plugin.Env) (plugin.Result, error) {
pg := p.Page
live, err := env.Get(ctx, pg.URL)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking (please confirm): providers/salesforce.md says "The live page 403s a plain fetch (bot-protected), so it's read through a Wayback Machine snapshot". But Run reads the live page first, and only calls Wayback afterwards to cite it. Wayback isn't a fallback here. If env.Get gets a 403 from patch-it's hosted ingestion service, this returns an error on every run and the plugin never emits anything.

Could you check one of these?

  • env.Get (the fetch layer) really does get a 200 from this URL, and the body it gets holds the <tbody>/<tr>/<td> table that Parse expects. developer.salesforce.com/docs/... pages may load their content client-side, and the saved fixture is a trimmed copy, so the golden test can't tell us this. Then fix the provider page's sentence.
  • Or, if the live page really does 403, the plugin needs a source it can actually fetch, and the provider page should say which one.

The same question applies to kit.Wayback's id_ snapshot: Cite re-parses the raw archived HTML and compares it to want. The fixture README says the Wayback rewrite prefixes were removed by hand, so please check the raw id_ body parses the same way.

Comment thread providers/salesforce.md Outdated

## Why this source

- **Source:** [API End-of-Life Policy](https://developer.salesforce.com/docs/platform/api-rest/guide/api-rest-eol.html), Salesforce's own table of REST API version ranges and their support status. The live page 403s a plain fetch (bot-protected), so it's read through a Wayback Machine snapshot, same as `retirements` and `feed`. Checked 2026-11-20 (page crawled 2026-10-31).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't match the code: retirements, feed and eol all env.Get the live page first, and they snapshot it on Wayback only to cite it. They never read through Wayback in place of the live page. If the live page 403s, Run fails before Wayback is called (see the comment on plugins/eol/eol.go:171). Please reword this once the fetch question there is settled.

@AbdullahT01

Copy link
Copy Markdown
Contributor Author

Good catch — this wasn't just a wording mismatch, the fetch itself was broken: env.Get always hits Salesforce's 403, and Run returned an error before ever reaching kit.Cite's snapshot. Confirmed the live page 403s every plain fetch I tried (including a browser User-Agent), but the Wayback Machine's own save-page-now request succeeds against it right now. Fixed Run to fall back to a fresh Wayback snapshot (used directly as both content and citation) when the live fetch is blocked with a 4xx, added two tests for it (the fallback firing on a 403, and not firing on a transient 503), and reworded the provider page to describe what the code actually does. @claude review

@claude

claude Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

Claude finished @AbdullahT01's task in 1m 9s —— View job


⚠️ Merge after fixes. The fallback fixes the crash I flagged, and the two new tests cover it well. One question from last time is still open, though: does the snapshot the Wayback Machine actually takes contain the table that Parse reads? Without that answer, we still can't be sure this plugin will ever emit anything in production.

Thanks for the quick and careful follow-up, @AbdullahT01. You were right that the problem was the fetch itself, not just the wording, and the fix is narrow: it only falls back on a 4xx, never on a 5xx or a timeout. 🙏

  • Read the new commit (185cca0) and the code it calls (kit.Wayback, fetch.StatusError, kittest.Archive, plugin.Env)
  • Checked it against AGENTS.md
  • Post the review

Rules

Rule Status Evidence
Plugin list (Rule 1) ✅ Unchanged since my last review. The salesforce row in docs/plugins.md still matches the plugin and the provider page.
Provider page ✅ providers/salesforce.md:11 now says what the code does: a direct GET first, then a fresh snapshot used both to read and to cite. There is also a new Known gap about paying for one snapshot on every run.
Golden test + quote ✅ Unchanged. 34 events, and testdata/README.md has the URL, date, license and quote.
Drop test ✅ TestDropsAStillSupportedBand is unchanged.
Every event cites a source ✅ On the fallback path, SourceURL is the snapshot URL the archive returns. TestFallsBackToTheArchiveWhenTheLiveFetchIs403d checks that it's never the blocked live URL.
Fetch only via env ✅ The direct read still goes through env.Get, and the fallback goes through kit.Archiver. plugins/eol imports fetch only for the StatusError type, not to fetch anything. No net/http, no filesystem.
Precision over recall ⚠️ When the fallback runs, kit.Cite's cross-check is skipped. That's fine, because the events are read from the same bytes they cite. What's still unproven is whether those bytes contain the table at all (see below). If they don't, the plugin fails safely: Parse returns an error and nothing is emitted.
Nothing private (Rule 2) ✅ Nothing private in the new commit. It is signed off.

Blocking

  1. Please confirm that a real id_ snapshot parses. This was the second half of my last comment, and the new commit doesn't answer it yet.

    • Why it matters: the fallback is now the only way this plugin gets its content in production. It reads the raw id_ capture through kit/wayback.go:78.
    • Why the tests can't tell: they feed kittest.Archive the hand-trimmed fixture, whose Wayback rewrite prefixes were removed by hand.
    • What could go wrong: "Save Page Now succeeds" only tells us the archive returned a snapshot URL. It doesn't tell us that what it stored is the rendered table. The archive may also have recorded Salesforce's 403 page, or a page that only fills in its content in the browser.
    • Evidence that would settle it:
      • one snapshot URL from 2026-11-20;
      • confirmation that running Findings on its id_ body gives the same 34 non-dropped findings as the golden file. A throwaway go run or a test you don't commit is enough.
    • If the snapshot doesn't contain the table, this source can't be read in production. Better to find that out before merge than in the hosted service's logs.
  2. The fallback depends on how the hosted service's env.Get reports errors (plugins/eol/eol.go:262).

    • The condition: liveOrArchived only falls back when errors.As(err, &*fetch.StatusError) matches.
    • What plugin.Env.Get promises: only func(ctx, url) ([]byte, error) (plugin/plugin.go:43). Nothing says its errors are, or wrap with %w, a *fetch.StatusError.
    • If they aren't: say patch-it's hosted ingestion service wraps the error with %v, or uses another client. Then the fallback never runs, and we're back to the error on every run that you just fixed.
    • Two ways to fix it: confirm the hosted Get returns fetch.Get / fetch.Cache.Get errors with %w, and say so in Env.Get's doc comment so the contract is written down. Or have this plugin not depend on it.

    Fix this →

Non-blocking (optional)

  • Narrow the fallback to 401/403. Any 4xx triggers it today, but the comment and the provider page only argue for the bot block.
    • A 404 or 410 would mean the page moved. Asking the archive to snapshot a missing page then wastes a Save Page Now call, and the error message hides the real cause behind "and so did asking the Wayback Machine to".
    • A 429 is the "transient" case the doc comment says it avoids hiding.
    • Something like se.StatusCode == 401 || se.StatusCode == 403 would match the doc comment exactly. Fix this →
  • Baseline churn. On the fallback path, the hash is taken from the archive's bytes. If Salesforce's page has a nonce or timestamp in its HTML, kit.Unchanged will never match, so a new baseline is saved on every run. No events are duplicated, because each event's fingerprint doesn't change. Still, it would be worth adding to the Known gap you wrote.
  • Save Page Now rate limits. One Save Page Now call per run is fine at the hosted service's schedule. But if it runs often, the archive may start answering 429 or returning an earlier capture. Earlier captures are still real and dated, so this isn't a precision problem; it may only be worth a sentence.
  • The three suggestions from my last review (copy the label from the cell for the excerpt, add a comment on the version regex, use <td[^>]*>) still apply. All optional.

Verified

  • I read the 185cca0 diff against e528a45, plus kit/wayback.go, kit/kittest/kittest.go, fetch/fetch.go and plugin/plugin.go.
  • Not run: go build, go test and the other make targets needed approval in this sandbox this time, and I couldn't fetch Salesforce or the archive. CI_go runs all of them on this push.

@AbdullahT01

Copy link
Copy Markdown
Contributor Author

Both answered.

Does a real id_ snapshot parse? Yes — just tested it live, not with the committed fixture:

  • Triggered a real save-page-now request against the exact page URL just now: https://web.archive.org/save/https://developer.salesforce.com/docs/platform/api-rest/guide/api-rest-eol.html → 302 to a brand-new capture, https://web.archive.org/web/20261120050620/....
  • Fetched that capture's raw id_ body exactly the way kit/wayback.go:78 does (same URL pattern, gzip-decoded).
  • Ran this plugin's own Findings on those bytes in a throwaway go run (not committed): 34 kept, 1 dropped — the same count as the golden file, same versions, same change types. The archive didn't record a 403 page or an unrendered shell; it has the real table.

Does plugin.Env.Get actually hand back a *fetch.StatusError? Traced it instead of assuming it: fetch.Cache.Get → get → attemptGet, and a non-2xx/304 response becomes &StatusError{URL, StatusCode} returned unwrapped at every one of those layers (no further wrapping to break errors.As). kit.go's own package comment already said plugin.Env.Get is fetch.Cache.Get in the hosted service; I've now written that contract onto Env.Get itself (plugin/plugin.go) instead of leaving it implicit in a different package's comment, per your first option.

Also addressed everything non-blocking from both reviews:

  • Narrowed the fallback to 401/403 only (not every 4xx): a 404/410 now fails normally instead of wasting a Save Page Now call on a page that moved, and a 429 falls through to the "transient" path as intended. Added test cases for 404/429/503 (no fallback) and both 401 and 403 (fallback).
  • The excerpt's label now comes from the cell's own text (r.Label, captured in Parse) instead of being rebuilt with fmt.Sprintf, so it stays word-for-word even if Salesforce's wording changes.
  • Commented the version regex's whole-number assumption.
  • cell now tolerates a <td class="…"> attribute.
  • Added two more Known gaps: baseline churn if a future capture ever varies by bytes the table doesn't (hash-only "unchanged" check, no event would duplicate since fingerprints are unaffected), and Wayback's own rate limits if this ran far more often than daily.

go build, go vet, gofmt -l, go test -race ./..., make imports, make nethttp, and the leak guard all pass locally on cf9887b.

@claude review

@claude

claude Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

Claude finished @AbdullahT01's task in 35s —— View job


✅ Ready to merge. Both blocking questions from my last review are answered, and every optional suggestion was taken. I found one small inaccuracy in a doc comment; it's optional and listed below.

Thank you, @AbdullahT01. Testing a fresh Save Page Now capture end to end, instead of trusting the trimmed fixture, settles the question that mattered most. Tracing the error path through fetch and then writing the contract onto Env.Get was the right fix too. 🙏

  • Read the new commit (cf9887b) against 185cca0
  • Checked the *fetch.StatusError path through fetch/fetch.go and fetch/cache.go
  • Checked it against AGENTS.md
  • Post the review

Rules

Rule Status Evidence
Plugin list (Rule 1) ✅ No change since my last review. The salesforce row in docs/plugins.md still matches the plugin and the provider page, and its links are absolute.
Provider page ✅ providers/salesforce.md has a complete Why this source and describes the fallback accurately. Two new Known gaps cover baseline churn and Save Page Now rate limits.
Golden test + quote ✅ testdata/golden/salesforce.json is unchanged. The label now comes from the cell (Row.Label), and the golden file didn't need an edit, so the cell text matches the old rebuilt label word for word. testdata/README.md has the URL, date, license and quote.
Drop test ✅ TestDropsAStillSupportedBand is unchanged.
Every event cites a source ✅ SourceURL is always a snapshot URL, from kit.Cite or from the fallback's own capture. SourceExcerpt is now the cell's own label plus its status.
Fetch only via env ✅ Direct reads go through env.Get and the fallback goes through kit.Archiver. fetch is imported only for the StatusError type. No net/http, no filesystem.
Precision over recall ✅ You confirmed a real id_ capture from 2026-11-20 gives the same 34 kept and 1 dropped findings as the golden file. The fallback now runs only on 401/403. A 404, 429 or 5xx is returned as an error, never shown as a page.
Nothing private (Rule 2) ✅ Nothing private in the new commit, and all three PR commits are signed off.

Blocking

None.

  • Blocking item 1 (does the id_ snapshot parse?) is resolved by your live test: a new capture at 20261120050620, its raw id_ body, and Findings giving 34 kept and 1 dropped findings.
  • Blocking item 2 (does env.Get return a *fetch.StatusError?) is resolved. I followed the same path:
    • fetch.go:127 returns &StatusError{…} without wrapping it.
    • fetch.go:52 does the same.
    • cache.go:59 passes err through as is.
    • So errors.As finds it, and plugin/plugin.go now documents the contract.

Non-blocking (optional)

  • Env.Get's doc comment overstates the test kit (plugin/plugin.go:43). It says that in the test kit Get "is fetch.Cache.Get (kit/kittest's Pages.Get wraps saved files the same way)".
    • In fact, kittest.Pages.Get (kit/kittest/kittest.go:41) reads files directly and never returns a *fetch.StatusError.
    • Your eol tests return one from a hand-written Get, which is fine.
    • A small rewording would keep the contract accurate: "In the hosted ingestion service, this is fetch.Cache.Get: a non-2xx, non-304 response fails with a *fetch.StatusError… Test kits that simulate a status should return the same type." Fix this →
  • Follow-up, out of scope for this PR: cmd/new-plugin --format eol isn't wired up yet, as you noted in the PR body. A follow-up issue would help the next provider that uses this format.

Verified

  • Read the cf9887b diff, fetch/fetch.go, fetch/cache.go, kit/kittest/kittest.go and the current plugins/eol/eol.go.
  • Not run by me: go build, go test and the make targets. CI_go runs all of them on this push. You reported they pass locally on cf9887b.
  • Not checked by me: the live Wayback capture, because my sandbox can't make network requests. I've relied on the test you described.

@adssib
adssib merged commit 3622775 into main Nov 20, 2026
3 of 4 checks passed
@adssib
adssib deleted the salesforce-source branch November 20, 2026 21:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

I15.2 · Watch Salesforce

2 participants