Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/writing-a-plugin.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ Write the numbers on the provider page, as `providers/vercel.md` does for its sp
| Kind | Reads | Add a source by |
|---|---|---|
| `plugins/openapi` | An OpenAPI spec per release, diffed with `core/diff/` and turned into events by `core/normalize/` | An `adapter.Adapter` in `plugins/openapi/adapter/` (latest tag, fetch a tag's spec; `Versions` to list every release) and a `plugin.RegisterProvider` block |
| `plugins/notices` | A live page of dated notices: a model provider's deprecations page (a table of date, model, replacement: Markdown or Markdoc), a vendor's RSS or Atom feed, a vendor's table of whole API version ranges and their support status | A `retire.Page` in `plugins/notices/retire/providers.go`, a `feed.Feed` with a reader for that vendor's entries in `plugins/notices/feed/`, or an `eol.Page` in `plugins/notices/eol/` (`new-plugin` doesn't scaffold this reader yet) |
| `plugins/notices` | A live page of dated notices: a model provider's deprecations page (a table of date, model, replacement: Markdown, Markdoc or HTML), a vendor's RSS or Atom feed, a vendor's table of whole API version ranges and their support status | A `retire.Page` in `plugins/notices/retire/providers.go`, a `feed.Feed` with a reader for that vendor's entries in `plugins/notices/feed/`, or an `eol.Page` in `plugins/notices/eol/` (`new-plugin` doesn't scaffold this reader yet) |
| `plugins/migrationguide` | A guide that lists what a release removed, at the release's tag: an SDK's migration guide per new major (removed request parameters), or a package's table of removed members (an RST simple table or a Markdown pipe table) | A `sdkguide.Guide` in `plugins/migrationguide/sdkguide/`, or a `members.Guide` in `plugins/migrationguide/members/` |
| `plugins/changelog` | A changelog: a `CHANGELOG.md` / `CHANGES.rst` / `HISTORY.rst`, each release's section at its tag (releases from PyPI or npm), or a vendor's machine-readable API changelog in JSON (per day, endpoint and API version, oasdiff change codes), commit by commit | A `changelog.File` with its `TagPrefix`, or a `changelog.Log` |
| `plugins/catalog` | A provider's own structured list of what it offers: a command catalog kept as one JSON file per command in its repo, read at each release tag and compared for commands that disappeared entirely | A `catalog.Commands` (repo, directory, release tag pattern); a catalog of another shape is a new reader type in the package |
Expand Down
18 changes: 17 additions & 1 deletion plugins/notices/google_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -66,8 +66,24 @@ func TestGeminiDropsModelsWithNoDateAndAgents(t *testing.T) {
}
}

// The pinned drop test for Vertex AI: a GA model's "June 28, 2027 or
// later" is how long it's kept at least, not a retirement. The Gemini image
// models' table has four such rows and one dated retirement
// (gemini-2.5-flash-image, March 15, 2027): one event. The same table with
// the dates plain would give five.
func TestVertexDropsAMinimumLifetime(t *testing.T) {
ga := section(t, "retire/testdata/vertex-model-versions.html", `<h3 id="gemini-image-models"`, `<h3 id="gemini-omni-models"`)
if n := readGoogle(t, retire.Vertex, ga); n != 1 {
t.Errorf("%d events, want the one dated retirement only", n)
}
plain := strings.ReplaceAll(ga, " or later", "")
if rs, err := retire.Parse(retire.Vertex.Format.Pipes([]byte(plain)), retire.Vertex.Keep); err != nil || len(rs) < 2 {
t.Errorf("the same table with plain dates: %d, %v; the drop proves nothing", len(rs), err)
}
}

// Google's Gemini dates are the earliest a model may go: the description
// says so.
// says so, and an exact table (Vertex AI's) doesn't.
func TestGeminiDatesAreTheEarliest(t *testing.T) {
saved := kittest.Pages{retire.Gemini.URL: "retire/testdata/gemini-deprecations.md"}
for _, e := range kittest.Page(t, kittest.Env(t, saved), Plugin{Page: retire.Gemini, Archiver: &kittest.Archive{T: t, Pages: saved}}) {
Expand Down
6 changes: 6 additions & 0 deletions plugins/notices/notices.go
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,12 @@ func init() {
URLs: []string{"https://ai.google.dev/gemini-api/docs/deprecations", "https://aistudio.google.com/docs/deprecations"},
Packages: []string{"pypi:google-genai", "npm:@google/genai"},
Plugin: Plugin{Page: retire.Gemini, Archiver: kit.Wayback{}},
}, {
ID: "vertex-model-retirements", Layer: plugin.LayerAIModels, Page: "providers/google-vertex-model-retirements.md",
Watches: "Vertex AI (Gemini Enterprise Agent Platform) model retirement dates and replacements, from Google Cloud's model versions tables",
URLs: []string{"https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/model-versions"},
Packages: []string{"pypi:google-genai", "pypi:google-cloud-aiplatform", "npm:@google/genai"},
Plugin: Plugin{Page: retire.Vertex, Archiver: kit.Wayback{}},
}},
})
plugin.RegisterProvider(plugin.Provider{
Expand Down
40 changes: 40 additions & 0 deletions plugins/notices/retire/formats.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
package retire

import (
"html"
"regexp"
"strconv"
"strings"
)

Expand Down Expand Up @@ -50,6 +52,44 @@ func markdocPipes(page []byte) []byte {
return []byte(strings.Join(out, "\n") + "\n")
}

var (
htmlTable = regexp.MustCompile(`(?is)<h([1-6])[^>]*>(.*?)</h[1-6]>|<table[^>]*>(.*?)</table>`)
htmlRow = regexp.MustCompile(`(?is)<tr[^>]*>(.*?)</tr>`)
htmlCell = regexp.MustCompile(`(?is)<t[hd][^>]*>(.*?)</t[hd]>`)
htmlCode = regexp.MustCompile(`(?is)<code[^>]*>(.*?)</code>`)
htmlTag = regexp.MustCompile(`<[^>]+>`)
htmlNewline = regexp.MustCompile(`\s+`)
)

// htmlPipes reads an HTML page's tables into pipe tables, each under the
// heading above it; <code> becomes a code span, every other tag goes.
func htmlPipes(page []byte) []byte {
var out []string
for _, m := range htmlTable.FindAllStringSubmatch(string(page), -1) {
if m[1] != "" {
level, _ := strconv.Atoi(m[1])
out = append(out, strings.Repeat("#", level)+" "+htmlText(m[2]))
continue
}
var rows [][]string
for _, r := range htmlRow.FindAllStringSubmatch(m[3], -1) {
var row []string
for _, c := range htmlCell.FindAllStringSubmatch(r[1], -1) {
row = append(row, htmlText(htmlCode.ReplaceAllString(c[1], "`$1`")))
}
if len(row) > 0 {
rows = append(rows, row)
}
}
out = append(out, pipeTable(rows)...)
}
return []byte(strings.Join(out, "\n") + "\n")
}

func htmlText(s string) string {
return strings.TrimSpace(htmlNewline.ReplaceAllString(html.UnescapeString(htmlTag.ReplaceAllString(s, " ")), " "))
}

// pipeTable writes rows (the first the header) as a Markdown pipe table,
// escaping a pipe inside a cell.
func pipeTable(rows [][]string) []string {
Expand Down
15 changes: 15 additions & 0 deletions plugins/notices/retire/providers.go
Original file line number Diff line number Diff line change
Expand Up @@ -71,3 +71,18 @@ var Gemini = Page{
Keep: googleModel.MatchString,
Earliest: true,
}

// Vertex is Google Cloud's Vertex AI model versions page (checked
// 2026-11-21): HTML tables of | Model ID | Release date | Retirement date |
// Replacement model ID | (or Recommended upgrade). A GA model's date is a
// minimum lifetime ("October 13, 2027 or later", "No sooner than May 20,
// 2028"), not a notice, and isn't read as a date; a retired or retiring
// model's is a plain date ("June 30, 2026").
var Vertex = Page{
API: "google/vertex-model-retirements",
Provider: "google",
Vendor: "Google",
URL: "https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/model-versions",
Format: FormatHTML,
Keep: googleModel.MatchString,
}
10 changes: 7 additions & 3 deletions plugins/notices/retire/retire.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@
// A page is a Markdown pipe table (OpenAI's and Anthropic's pages, served as
// markdown at their URL plus ".md", checked 2026-10-13), a Markdoc table
// (Google's Gemini API page, {% table %} with one "* cell" per line, checked
// 2026-11-21), read into pipe rows first, with its headings, so one parser
// reads every page. A table is a retirement
// 2026-11-21), or an HTML table (Google Cloud's Vertex AI model versions,
// checked 2026-11-21). The last two are read into pipe rows first, with
// their headings, so one parser reads every page. A table is a retirement
// table when it has a shutdown or retirement date column, a model column and
// a replacement column, in any order. The event cites a Wayback Machine
// snapshot of the page (kit/), never the live page: the contract requires a
Expand All @@ -29,7 +30,7 @@ type Page struct {
API string // the source id the events carry: "openai/models"
Provider string // the model subject's provider: "openai"
Vendor string // for sentences: "OpenAI"
URL string // the page, as markdown
URL string // the page, as markdown or HTML
// Format is how the page writes its tables: Markdown pipe tables (the
// default), Markdoc or HTML.
Format Format
Expand All @@ -48,6 +49,7 @@ type Format string
const (
FormatMarkdown Format = ""
FormatMarkdoc Format = "markdoc"
FormatHTML Format = "html"
)

// Retirement is one model name the provider stops serving on Date, with the
Expand All @@ -72,6 +74,8 @@ func (f Format) Pipes(page []byte) []byte {
switch f {
case FormatMarkdoc:
return markdocPipes(page)
case FormatHTML:
return htmlPipes(page)
}
return page
}
Expand Down
3 changes: 2 additions & 1 deletion plugins/notices/retire/testdata/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

The two providers' deprecations pages, as markdown (each page's URL plus `.md`). `retire`'s tests and the `openai-models` / `anthropic-models` golden tests (`plugins/notices`) read them.

**License:** excerpts for testing; © OpenAI and © Anthropic respectively. Not covered by this repo's license. Google's page is © Google, CC BY 4.0 (Google's developer documentation).
**License:** excerpts for testing; © OpenAI and © Anthropic respectively. Not covered by this repo's license. Google's two pages are © Google, CC BY 4.0 (Google's developer documentation).

**Trimmed.** Each file keeps the page title and, for every retirement table that names at least one model the plugin keeps (text models code sends), the section heading above it and the whole table, copied unchanged. Prose, notice-period policy, endpoint tables and tables of only image, audio or embedding models were left out. `retire.Parse` returns exactly the same retirements from the trimmed and the full pages, so the golden files didn't change.

Expand All @@ -11,3 +11,4 @@ The two providers' deprecations pages, as markdown (each page's URL plus `.md`).
| `openai-deprecations.md` | https://developers.openai.com/api/docs/deprecations.md | 2026-10-13 | "\| October 23, 2026 \| `gpt-4-0613` \\\| `gpt-4`, … \| `gpt-5.6-sol` \|" |
| `anthropic-deprecations.md` | https://platform.claude.com/docs/en/about-claude/model-deprecations.md | 2026-10-13 | "\| November 30, 2026 \| `claude-sonnet-4-5-20250929` \| `claude-sonnet-5-5` \|" |
| `gemini-deprecations.md` | https://aistudio.google.com/docs/deprecations (where https://ai.google.dev/gemini-api/docs/deprecations redirects; served as Markdown). Kept whole, byte for byte | 2026-11-21 | "\| `gemini-2.0-flash` \| February 5, 2025 \| June 1, 2026 \| `gemini-3.6-flash` \|", as Markdoc `* cell` lines |
| `vertex-model-versions.html` | https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/model-versions. Excerpt: every `<h2>`/`<h3>` line and every line of every `<table>` (982 of 10,074 lines), unchanged; scripts, navigation and prose left out. `retire.Parse` finds the same 39 retirements in the excerpt as in the full page | 2026-11-21 | "&lt;td&gt;June 30, 2026&lt;/td&gt;" in the "Retired models" table |
Loading
Loading