From 009366b537a78f1627e6f81122c243057eb60c4e Mon Sep 17 00:00:00 2001 From: tannevaled Date: Tue, 3 Nov 2026 15:21:03 +0100 Subject: [PATCH 1/2] Ruby 4.0.9: the figure, the version, and a limitation that is now resolved MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit rbgo reports RUBY_VERSION 4.0.9 (ruby#882) and BASELINE was re-recorded at that version, on the pinned runner that enforces it: 23,511 -> 23,673. ## ⛔ My grep found one of four, and the tool's ARITHMETIC found the rest Searching content for `23,511` and `23511` returned a single hit, and that reads exactly like "one place to fix". `tools/conformancefigure` then said: 4 total claim(s) and 2 version claim(s) checked ... content/en/_index.md:13: claims 23673, the gate records 23511 Four claims checked, ONE disagreeing -- so three claims still agreed with the old figure, in a file I had just declared clean. They are written with a SPACE, `23 511`, which neither of my patterns could match: content/en/conformance.md:209 content/en/conformance.md:262 content/en/roadmap.md:22 What caught it was not re-reading the page. It was a count that refused to add up: 4 checked against 1 flagged. A tool that reports its DENOMINATOR makes a narrow search self-refuting; one that printed only its findings would have agreed with me. ## The known-limitation row is resolved, not deleted The `RUBY_VERSION` row asked for "a CI sweep that moves `CUR_VERSION` without regenerating the baseline, so the ratchet names every file that moves in both directions". That sweep was run. The row now carries its result rather than vanishing, including the two cautions the numbers need: - `+144` is a NET, and `13 regressed` is not 13 defects: six of the thirteen lost only examples the corpus stops asking for, each with a CLOSED guard in the census; - `core/objectspace/_id2ref_spec.rb` reads 10 -> 0 and that is eleven examples below 4.0 against ONE above it, not ten regressions. The 27 examples that WENT RED are the move's real remaining work, and no pass total shows them. ## Also stated `CUR_VERSION = RUBY_VERSION`, so the shim cannot disagree with the interpreter -- the "two copies, agreeing" wording was true before ruby#845 and is no longer the situation. And the baseline was re-recorded on the pinned runner that enforces it, not on a laptop: one laptop reads +9 against CI for no change in the code. ⛔ This cannot land before ruby#882. Verified, not assumed: run against main today the checker prints all six disagreements and exits 1. Co-Authored-By: Claude Opus 5 --- content/en/_index.md | 12 +++++++----- content/en/conformance.md | 23 ++++++++++++++--------- content/en/roadmap.md | 2 +- 3 files changed, 22 insertions(+), 15 deletions(-) diff --git a/content/en/_index.md b/content/en/_index.md index 8449d16..eb06455 100644 --- a/content/en/_index.md +++ b/content/en/_index.md @@ -10,13 +10,15 @@ the lexer, parser and compiler embedded in the binary. Embed it in a Go program, ship it as a single static executable that cross-compiles wherever Go does, with no C toolchain and no libruby. -**How complete is it?** It passes **23,511** of ruby/spec's `language/` + `core/` +**How complete is it?** It passes **23,673** of ruby/spec's `language/` + `core/` examples — a large and growing subset of Ruby, and **not** a drop-in replacement for CRuby. That count is taken with ruby/spec's own version guards set to resolve at -Ruby **3.4.1**, the version this runtime reports: every example behind a -`ruby_version_is "4.0"` guard is removed from the suite before anything is -counted, so the figure describes 3.4-shaped behaviour even though the project -targets 4.0 semantics — 61 block guards and 5 inline sites sit at that boundary +Ruby **4.0.9**, the version this runtime reports and the latest MRI release: the +examples behind a `ruby_version_is "4.0"` guard are counted, and those behind +`ruby_version_is ""..."4.0"` are no longer asked for. The move from 3.4.1 was +measured before it was made, both engines built from one commit minus the +version string: **+144 passing**, and **27 examples that went red** — a figure +no pass total shows ([ruby#799](https://github.com/go-embedded-ruby/ruby/issues/799)). [Conformance]({{< relref "conformance.md#where-it-stands-measured-2026-09-26" >}}) states exactly what that counts and what it leaves out, and diff --git a/content/en/conformance.md b/content/en/conformance.md index fdddab8..01ba835 100644 --- a/content/en/conformance.md +++ b/content/en/conformance.md @@ -154,12 +154,17 @@ workflow enforces it on every push and pull request. ### Where it stands — measured 2026-09-26 {#where-it-stands-measured-2026-09-26} {{< callout type="info" >}} -**The gate's current total is **23 511**** +**The gate's current total is **23 673**** -Counted with ruby/spec's version guards set to resolve at Ruby **3.4.1** — the -version `RUBY_VERSION` reports and the version the shim measures at. That -selects the population before any counting happens, so the total is not a -percentage of Ruby 4.0; see [Known limitations](#known-limitations). +Counted with ruby/spec's version guards set to resolve at Ruby **4.0.9** — the +version `RUBY_VERSION` reports and the version the shim measures at, since +`CUR_VERSION = RUBY_VERSION`. That selects the population before any counting +happens, so the total is still not a percentage of Ruby; see +[Known limitations](#known-limitations). + +The baseline was re-recorded at 4.0.9 **on the pinned runner that enforces +it**, not on a developer's machine — one laptop reads +9 against CI for no +change in the code at all. The table below is one **dated full sweep**, and a dated measurement going stale is not an error: it is what it says it is. The number that moves with @@ -168,7 +173,7 @@ quote, and it is read straight out of the gate's own file: ```console $ awk -F'\t' '/^OK\t/{n+=$3} END{print n}' scripts/conformance/rubyspec/BASELINE -23511 +23673 ``` The breakdown below — failing, erroring, skipped — needs a full sweep to @@ -201,7 +206,7 @@ same ten files producing no result. level **CI guarantees**; a measured total is **what rbgo did on a given run**. Quoting the guaranteed figure as the conformance result understates rbgo by whatever has landed since the record was last written; quoting a measured total -as a guarantee overstates it by the same amount. The sum is **23 511**, and +as a guarantee overstates it by the same amount. The sum is **23 673**, and both the repository README and this page read that figure out of the file mechanically, each with a check that fails if they disagree — which is why neither carries a release number beside it. The figure is verified @@ -254,7 +259,7 @@ Both are gains in what the measurement can **see**, not in what the VM can **do**. The rest of the climb is the VM: the floor went from **6 000** when the ratchet landed on 2026-08-03 ([#263](https://github.com/go-embedded-ruby/ruby/pull/263)) to **22 475** by the -time it was replaced by the per-file baseline, whose sum is **23 511**. +time it was replaced by the per-file baseline, whose sum is **23 673**. ### Known limitations {#known-limitations} @@ -300,7 +305,7 @@ the call returns successfully and nothing says the guarantee is missing: | | | | --- | --- | -| `RUBY_VERSION` reports `"3.4.1"`, though the differential oracle is MRI 4.0.7, the pinned corpus is 4.0.x, and the latest MRI release is 4.0.9 (2026-10-23). That is not only cosmetic: the shim's `CUR_VERSION` is pinned to the same `3.4.1`, so **61 block guards and 5 inline sites at the `ruby_version_is "4.0"` boundary never run**, while **41 guards that end *before* 4.0 are being asserted**. Guard counts are exact, censused by `scripts/conformance/rubyspec/cmd/guards`; the example count behind them is not stated here because a static scan cannot expand `it_behaves_like`, and the figure of 167 quoted when this was first reported does not reproduce — it needs a CI sweep that moves `CUR_VERSION` without regenerating the baseline, so the ratchet names every file that moves in both directions | [#799](https://github.com/go-embedded-ruby/ruby/issues/799) | +| **Resolved 2026-11-03.** `RUBY_VERSION` now reports `"4.0.9"`, the latest MRI release, and `CUR_VERSION = RUBY_VERSION` so the shim cannot disagree with it. The sweep this entry asked for was run — both engines built from **one commit minus the version string**, over `core` and `language` together: **+144 passing, 45 files moved, 27 examples that WENT RED**. The 27 are the move's real remaining work and are listed per item on the issue. Two cautions the figures carry: `+144` is a **net** and `13 regressed` is not 13 defects — six of the thirteen lost only examples the corpus stops asking for, each with a `CLOSED` guard in the census, and `core/objectspace/_id2ref_spec.rb` reading `10 → 0` is eleven examples below 4.0 against **one** above it | [#799](https://github.com/go-embedded-ruby/ruby/issues/799) | | `Errno` carries all **158** of MRI's constant names, but **32** report errno `0` where MRI has a real number (`EAUTH`, `EBADRPC`, `EDEVERR`, …). By set difference on darwin: 81 names are zero-valued under rbgo, 49 under MRI, and every MRI zero is also zero under rbgo | [#633](https://github.com/go-embedded-ruby/ruby/issues/633) | | `Thread#backtrace` answers for the **current** thread only; another thread raises `NotImplementedError` | — | | `Process.fork` **does not exist** — Go's runtime cannot be forked safely | — | diff --git a/content/en/roadmap.md b/content/en/roadmap.md index 90a252c..9507997 100644 --- a/content/en/roadmap.md +++ b/content/en/roadmap.md @@ -19,6 +19,6 @@ building whole subsystems in isolation. There are nine phases, 0 through 8. | 5 | Full front-end | Complete the lexer/parser/compiler to full Ruby 4.0 syntax (or settle the Prism-on-wazero decision). **Landed:** string interpolation, keyword args, endless methods (`def foo = expr`), beginless/endless ranges, the stabby lambda, **multiple assignment / destructuring**, **full pattern matching `case`/`in`** (value/bind/class/array/hash/find/pin/alternative patterns, guards, one-line `=>`/`in`, `deconstruct`/`deconstruct_keys`), **safe navigation `&.`**, **numbered params (`_1`/`_2`) and `it`**, **heredocs** (`<<`/`<<-`/`<<~`, interpolating and literal), **`%w`/`%i` arrays**, and the **`%q`/`%Q`/`%W`/`%I` literals**. | **Done** (core) | | 6 | Standard library | Pure-Go stdlib coverage, with the regexp engine from the `go-ruby-regexp` sibling org. **Regexp bridge landed:** `/re/imx` literals + `Regexp`/`MatchData` (`source`/`match`/`match?`/`=~`/`===`, `[]` by index or name, `pre_match`/`post_match`/`captures`/`named_captures`/`begin`/`end`/`size`), `String#=~`/`match`/`match?`/`scan`, and `sub`/`gsub`/`split` over a Regexp (backref templates, blocks, captured-group interpolation, field limit). CGO=0 via `go-ruby-regexp`; differential-tested against MRI. **Also landed:** `Time`/`Date`/`BigDecimal`, `Set`/`Bag`, and the Go-native leaves `JSON` (generate/parse/pretty_generate + `to_json`), `Digest` (MD5/SHA1/SHA256/SHA512), `Base64`, **`File`** (path helpers + read/write/stat/delete, `Errno::ENOENT`), a **MRI-faithful `Random`** (seeded MT19937, bit-exact), **`Zlib`** (crc32/adler32 + Deflate/Inflate), a **byte-exact `Marshal`** (`dump`/`load`, on the pure-Go `go-ruby-marshal` engine), **`Thread`/`Mutex`/`Queue`** on an emulated GVL (one Ruby thread at a time, MRI memory model, race-free), **`IO`/`StringIO`** (`$stdout`/`$stderr`/`$stdin` as real objects; `puts`/`print`/`p` route through `$stdout` so reassigning it to a `StringIO` captures output) and **`Dir`** (entries/children/glob, mkdir/rmdir/chdir, `Errno::ENOENT`/`Errno::EEXIST`), and **`File.open` streams** (`File < IO`, modes `r`/`w`/`a`/`+`, block form auto-flush/close, `File.readlines`/`foreach`). | **Substantially landed** | | 7 | Build toolchain | `rbgo build`. **Landed:** ahead-of-time **method compilation** to native Go (bytecode → Go → machine code), with **unboxed `int64` integer kernels** that deopt to the interpreter on overflow/`÷0` — the generated `fib(30)` beats **MRI + YJIT ~4×** while staying correct for every input ([AOT compiler]({{< relref "architecture/build.md#ahead-of-time-method-compilation-landed" >}})). **Closed-world builds landed** (`rbgo build --closed`): the program is baked in as bytecode (and the prelude loads from frozen bytecode), the **lexer/parser/compiler are dropped** behind the `rbgo_closed` build tag, and a require-graph scan reports any `eval`/`require` that would then raise — yielding a smaller, self-contained binary that runs with no source file. (Per-class stdlib tree-shaking beyond the front-end was measured and skipped — the Go-implemented bindings are small pure-Go libraries, under 2% of a runtime-dominated binary; the front-end drop is the real saving.) | **Done** | -| 8 | Conformance & performance | ruby/spec conformance and representation/perf work (tagging, fast paths, allocation). **Started:** allocation cuts — **small-integer interning** (`object.IntValue` reuses a shared box for ints in [-256, 1024]; a small-int loop drops from ~245k allocs to 1) and **frame-environment recycling** (`exec` pools per-call `*Env`s via a capture-tracked free-list — fib's call allocations halve, ~14% faster). **Conformance & benchmark ladder:** a three-way differential oracle (rbgo / **MRI** / **JRuby**, `scripts/oracle.sh`), then pure-Ruby gem test suites as corpora (ActiveSupport `core_ext`), then real-world workloads. A heavyweight front-end sweep now accepts **~100% of all valid Ruby** across Rails + Puppet (**parse + compile** Rails 99.82% / Puppet 100%, 0 over-permissive), and on the runtime side **the real `puppet apply` CLI runs end-to-end** (Puppet 8.11.0): `require "puppet"` boots the framework and the genuine `Puppet::Util::CommandLine` → `Puppet::Application::Apply` path (real `OptionParser`, all types + providers, settings catalog applied to disk, then the user catalog via the transaction / RAL) applies a `notify` resource and emits real Puppet output (`Notice: hi from rbgo cli`), exiting 0. **Runtime conformance, since July:** the ruby/spec `language/` + `core/` suites run in CI against a **per-file** baseline, which names any spec file that regresses instead of absorbing it in a total; its sum is **23 511**. It replaced a single shrink-only floor that had climbed from **6 000** (2026-08-03) to 22 475 — a threshold needed a margin wider than the largest spec file, so it could not see a regression smaller than that — see [Conformance]({{< relref "conformance.md#where-it-stands-measured-2026-09-26" >}}). **Active next milestone:** real file/package/service resource *providers* (host convergence) + run-report persistence. | **In progress** | +| 8 | Conformance & performance | ruby/spec conformance and representation/perf work (tagging, fast paths, allocation). **Started:** allocation cuts — **small-integer interning** (`object.IntValue` reuses a shared box for ints in [-256, 1024]; a small-int loop drops from ~245k allocs to 1) and **frame-environment recycling** (`exec` pools per-call `*Env`s via a capture-tracked free-list — fib's call allocations halve, ~14% faster). **Conformance & benchmark ladder:** a three-way differential oracle (rbgo / **MRI** / **JRuby**, `scripts/oracle.sh`), then pure-Ruby gem test suites as corpora (ActiveSupport `core_ext`), then real-world workloads. A heavyweight front-end sweep now accepts **~100% of all valid Ruby** across Rails + Puppet (**parse + compile** Rails 99.82% / Puppet 100%, 0 over-permissive), and on the runtime side **the real `puppet apply` CLI runs end-to-end** (Puppet 8.11.0): `require "puppet"` boots the framework and the genuine `Puppet::Util::CommandLine` → `Puppet::Application::Apply` path (real `OptionParser`, all types + providers, settings catalog applied to disk, then the user catalog via the transaction / RAL) applies a `notify` resource and emits real Puppet output (`Notice: hi from rbgo cli`), exiting 0. **Runtime conformance, since July:** the ruby/spec `language/` + `core/` suites run in CI against a **per-file** baseline, which names any spec file that regresses instead of absorbing it in a total; its sum is **23 673**. It replaced a single shrink-only floor that had climbed from **6 000** (2026-08-03) to 22 475 — a threshold needed a margin wider than the largest spec file, so it could not see a regression smaller than that — see [Conformance]({{< relref "conformance.md#where-it-stands-measured-2026-09-26" >}}). **Active next milestone:** real file/package/service resource *providers* (host convergence) + run-report persistence. | **In progress** | See [Phase 0 — Vertical slice]({{< relref "phases/phase0.md" >}}) for what runs today. From 6c602c991a73a0fff17f9c8231afc25f0ada07e1 Mon Sep 17 00:00:00 2001 From: tannevaled Date: Wed, 4 Nov 2026 12:48:20 +0100 Subject: [PATCH 2/2] The figure is 23,672, and the gate could not see two of the lines claiming it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit core/random/bytes_spec.rb draws 8 or 9 passing examples from run to run. The 2026-11-03 re-record banked the 9 and the next ordinary CI run went red, so ruby#885 pinned that row to its floor: the gate's total is 23,672, not 23,673. Refreshing the figure exposed a second defect in conformancefigure itself. It reported "4 total claim(s) checked" on a tree that holds five, and "5" on the landing tree that holds six. The two it skipped were: **The gate's current total is **23 673**** (conformance.md:157) The sum of those rows is 23,673 (landing list.html:73) The first uses "total is", which the alternation did not list. The second puts two words between "sum" and "is", and the alternation spelled out only the three determiners met so far — "the sum is", "its sum is", "whose sum is". Neither line was ever a candidate, so both runs looked complete while leaving a stale figure on a published page. The verdicts the tool did reach were all correct. What made the gap visible was the denominator it prints next to them: 4 against a grep that found 5. A tool that printed only its findings would have agreed with the omission. claimBefore now accepts "(sum|total) is" and "(sum|total) of is", and both lines are vocabulary cases, so a future narrowing fails before it ships. Nothing speculative was added: every alternative is a phrasing one of the two sites uses today. The landing page's ratio line ("93.3% pass (22,488 of 24,116)") is already a must-miss case and still misses. Checked locally against both trees: docs 5/5 checked and clean, landing 6/6 checked and clean, against the gate's 23672 at Ruby 4.0.9. Co-Authored-By: Claude Opus 5 --- content/en/_index.md | 2 +- content/en/conformance.md | 6 +++--- content/en/roadmap.md | 2 +- tools/conformancefigure/main.go | 26 +++++++++++++++++++++++++- 4 files changed, 30 insertions(+), 6 deletions(-) diff --git a/content/en/_index.md b/content/en/_index.md index eb06455..ccbaa5d 100644 --- a/content/en/_index.md +++ b/content/en/_index.md @@ -10,7 +10,7 @@ the lexer, parser and compiler embedded in the binary. Embed it in a Go program, ship it as a single static executable that cross-compiles wherever Go does, with no C toolchain and no libruby. -**How complete is it?** It passes **23,673** of ruby/spec's `language/` + `core/` +**How complete is it?** It passes **23,672** of ruby/spec's `language/` + `core/` examples — a large and growing subset of Ruby, and **not** a drop-in replacement for CRuby. That count is taken with ruby/spec's own version guards set to resolve at Ruby **4.0.9**, the version this runtime reports and the latest MRI release: the diff --git a/content/en/conformance.md b/content/en/conformance.md index 01ba835..9849b87 100644 --- a/content/en/conformance.md +++ b/content/en/conformance.md @@ -154,7 +154,7 @@ workflow enforces it on every push and pull request. ### Where it stands — measured 2026-09-26 {#where-it-stands-measured-2026-09-26} {{< callout type="info" >}} -**The gate's current total is **23 673**** +**The gate's current total is **23 672**** Counted with ruby/spec's version guards set to resolve at Ruby **4.0.9** — the version `RUBY_VERSION` reports and the version the shim measures at, since @@ -206,7 +206,7 @@ same ten files producing no result. level **CI guarantees**; a measured total is **what rbgo did on a given run**. Quoting the guaranteed figure as the conformance result understates rbgo by whatever has landed since the record was last written; quoting a measured total -as a guarantee overstates it by the same amount. The sum is **23 673**, and +as a guarantee overstates it by the same amount. The sum is **23 672**, and both the repository README and this page read that figure out of the file mechanically, each with a check that fails if they disagree — which is why neither carries a release number beside it. The figure is verified @@ -259,7 +259,7 @@ Both are gains in what the measurement can **see**, not in what the VM can **do**. The rest of the climb is the VM: the floor went from **6 000** when the ratchet landed on 2026-08-03 ([#263](https://github.com/go-embedded-ruby/ruby/pull/263)) to **22 475** by the -time it was replaced by the per-file baseline, whose sum is **23 673**. +time it was replaced by the per-file baseline, whose sum is **23 672**. ### Known limitations {#known-limitations} diff --git a/content/en/roadmap.md b/content/en/roadmap.md index 9507997..eb5b95e 100644 --- a/content/en/roadmap.md +++ b/content/en/roadmap.md @@ -19,6 +19,6 @@ building whole subsystems in isolation. There are nine phases, 0 through 8. | 5 | Full front-end | Complete the lexer/parser/compiler to full Ruby 4.0 syntax (or settle the Prism-on-wazero decision). **Landed:** string interpolation, keyword args, endless methods (`def foo = expr`), beginless/endless ranges, the stabby lambda, **multiple assignment / destructuring**, **full pattern matching `case`/`in`** (value/bind/class/array/hash/find/pin/alternative patterns, guards, one-line `=>`/`in`, `deconstruct`/`deconstruct_keys`), **safe navigation `&.`**, **numbered params (`_1`/`_2`) and `it`**, **heredocs** (`<<`/`<<-`/`<<~`, interpolating and literal), **`%w`/`%i` arrays**, and the **`%q`/`%Q`/`%W`/`%I` literals**. | **Done** (core) | | 6 | Standard library | Pure-Go stdlib coverage, with the regexp engine from the `go-ruby-regexp` sibling org. **Regexp bridge landed:** `/re/imx` literals + `Regexp`/`MatchData` (`source`/`match`/`match?`/`=~`/`===`, `[]` by index or name, `pre_match`/`post_match`/`captures`/`named_captures`/`begin`/`end`/`size`), `String#=~`/`match`/`match?`/`scan`, and `sub`/`gsub`/`split` over a Regexp (backref templates, blocks, captured-group interpolation, field limit). CGO=0 via `go-ruby-regexp`; differential-tested against MRI. **Also landed:** `Time`/`Date`/`BigDecimal`, `Set`/`Bag`, and the Go-native leaves `JSON` (generate/parse/pretty_generate + `to_json`), `Digest` (MD5/SHA1/SHA256/SHA512), `Base64`, **`File`** (path helpers + read/write/stat/delete, `Errno::ENOENT`), a **MRI-faithful `Random`** (seeded MT19937, bit-exact), **`Zlib`** (crc32/adler32 + Deflate/Inflate), a **byte-exact `Marshal`** (`dump`/`load`, on the pure-Go `go-ruby-marshal` engine), **`Thread`/`Mutex`/`Queue`** on an emulated GVL (one Ruby thread at a time, MRI memory model, race-free), **`IO`/`StringIO`** (`$stdout`/`$stderr`/`$stdin` as real objects; `puts`/`print`/`p` route through `$stdout` so reassigning it to a `StringIO` captures output) and **`Dir`** (entries/children/glob, mkdir/rmdir/chdir, `Errno::ENOENT`/`Errno::EEXIST`), and **`File.open` streams** (`File < IO`, modes `r`/`w`/`a`/`+`, block form auto-flush/close, `File.readlines`/`foreach`). | **Substantially landed** | | 7 | Build toolchain | `rbgo build`. **Landed:** ahead-of-time **method compilation** to native Go (bytecode → Go → machine code), with **unboxed `int64` integer kernels** that deopt to the interpreter on overflow/`÷0` — the generated `fib(30)` beats **MRI + YJIT ~4×** while staying correct for every input ([AOT compiler]({{< relref "architecture/build.md#ahead-of-time-method-compilation-landed" >}})). **Closed-world builds landed** (`rbgo build --closed`): the program is baked in as bytecode (and the prelude loads from frozen bytecode), the **lexer/parser/compiler are dropped** behind the `rbgo_closed` build tag, and a require-graph scan reports any `eval`/`require` that would then raise — yielding a smaller, self-contained binary that runs with no source file. (Per-class stdlib tree-shaking beyond the front-end was measured and skipped — the Go-implemented bindings are small pure-Go libraries, under 2% of a runtime-dominated binary; the front-end drop is the real saving.) | **Done** | -| 8 | Conformance & performance | ruby/spec conformance and representation/perf work (tagging, fast paths, allocation). **Started:** allocation cuts — **small-integer interning** (`object.IntValue` reuses a shared box for ints in [-256, 1024]; a small-int loop drops from ~245k allocs to 1) and **frame-environment recycling** (`exec` pools per-call `*Env`s via a capture-tracked free-list — fib's call allocations halve, ~14% faster). **Conformance & benchmark ladder:** a three-way differential oracle (rbgo / **MRI** / **JRuby**, `scripts/oracle.sh`), then pure-Ruby gem test suites as corpora (ActiveSupport `core_ext`), then real-world workloads. A heavyweight front-end sweep now accepts **~100% of all valid Ruby** across Rails + Puppet (**parse + compile** Rails 99.82% / Puppet 100%, 0 over-permissive), and on the runtime side **the real `puppet apply` CLI runs end-to-end** (Puppet 8.11.0): `require "puppet"` boots the framework and the genuine `Puppet::Util::CommandLine` → `Puppet::Application::Apply` path (real `OptionParser`, all types + providers, settings catalog applied to disk, then the user catalog via the transaction / RAL) applies a `notify` resource and emits real Puppet output (`Notice: hi from rbgo cli`), exiting 0. **Runtime conformance, since July:** the ruby/spec `language/` + `core/` suites run in CI against a **per-file** baseline, which names any spec file that regresses instead of absorbing it in a total; its sum is **23 673**. It replaced a single shrink-only floor that had climbed from **6 000** (2026-08-03) to 22 475 — a threshold needed a margin wider than the largest spec file, so it could not see a regression smaller than that — see [Conformance]({{< relref "conformance.md#where-it-stands-measured-2026-09-26" >}}). **Active next milestone:** real file/package/service resource *providers* (host convergence) + run-report persistence. | **In progress** | +| 8 | Conformance & performance | ruby/spec conformance and representation/perf work (tagging, fast paths, allocation). **Started:** allocation cuts — **small-integer interning** (`object.IntValue` reuses a shared box for ints in [-256, 1024]; a small-int loop drops from ~245k allocs to 1) and **frame-environment recycling** (`exec` pools per-call `*Env`s via a capture-tracked free-list — fib's call allocations halve, ~14% faster). **Conformance & benchmark ladder:** a three-way differential oracle (rbgo / **MRI** / **JRuby**, `scripts/oracle.sh`), then pure-Ruby gem test suites as corpora (ActiveSupport `core_ext`), then real-world workloads. A heavyweight front-end sweep now accepts **~100% of all valid Ruby** across Rails + Puppet (**parse + compile** Rails 99.82% / Puppet 100%, 0 over-permissive), and on the runtime side **the real `puppet apply` CLI runs end-to-end** (Puppet 8.11.0): `require "puppet"` boots the framework and the genuine `Puppet::Util::CommandLine` → `Puppet::Application::Apply` path (real `OptionParser`, all types + providers, settings catalog applied to disk, then the user catalog via the transaction / RAL) applies a `notify` resource and emits real Puppet output (`Notice: hi from rbgo cli`), exiting 0. **Runtime conformance, since July:** the ruby/spec `language/` + `core/` suites run in CI against a **per-file** baseline, which names any spec file that regresses instead of absorbing it in a total; its sum is **23 672**. It replaced a single shrink-only floor that had climbed from **6 000** (2026-08-03) to 22 475 — a threshold needed a margin wider than the largest spec file, so it could not see a regression smaller than that — see [Conformance]({{< relref "conformance.md#where-it-stands-measured-2026-09-26" >}}). **Active next milestone:** real file/package/service resource *providers* (host convergence) + run-report persistence. | **In progress** | See [Phase 0 — Vertical slice]({{< relref "phases/phase0.md" >}}) for what runs today. diff --git a/tools/conformancefigure/main.go b/tools/conformancefigure/main.go index e748737..abdadcf 100644 --- a/tools/conformancefigure/main.go +++ b/tools/conformancefigure/main.go @@ -106,8 +106,24 @@ var ( // Thousands may be separated by a space (the docs' style) or a comma (the // landing page's and the README's), and the figure may be wrapped in Markdown // bold or HTML tags. +// +// "total is" and the "sum of is" shape were added on 2026-11-04. The +// run that refreshed the figure for 23,672 reported "4 total claim(s) checked" +// on a docs tree that held FIVE and "5 … checked" on a landing tree that held +// SIX. The two it could not see said "The gate's current total is 23 673" and +// "The sum of those rows is 23,673": the first used a word the alternation did +// not have, the second put two words between "sum" and "is", and the +// alternation spelled only the three determiners it had met so far ("the", "its", +// "whose"). Nothing was wrong with the verdicts it did reach; the stale lines +// were simply never candidates, and only the printed denominator -- against a +// grep of the same trees -- made the gap visible. A vocabulary list is a +// filter, so every phrasing missing from it is a figure the gate cannot refuse. +// +// Nothing speculative goes in this list: each alternative is a phrasing that +// exists in the two sites today, because a word added for a sentence nobody +// writes is a false positive waiting for the sentence that does. var ( - claimBefore = regexp.MustCompile(`(?i)\b(it passes|the sum is|its sum is|whose sum is|summing|passes)\b[^0-9]{0,24}\*{0,2}(\d{1,3}[ ,]?\d{3})\*{0,2}`) + claimBefore = regexp.MustCompile(`(?i)\b(it passes|passes|summing|(?:sum|total)\s+of\s+[a-z ]{1,24}\s+is|(?:sum|total)\s+is)\b[^0-9]{0,24}\*{0,2}(\d{1,3}[ ,]?\d{3})\*{0,2}`) claimAfter = regexp.MustCompile(`(?i)\**(\d{1,3}[ ,]?\d{3})\**(?:|)?[^.!?0-9]{0,90}?\bexamples?\s+(?:pass|passing)\b`) ) @@ -182,6 +198,14 @@ var vocabularyCases = []struct { {"**How complete is it?** It passes **23,511** of ruby/spec's `language/`", true, "23,511"}, {"baseline, whose sum is **23 511** at v0.3.0 (2026-10-11).", true, "23 511"}, {"as a guarantee overstates it by the same amount. The sum is **23 511** as of", true, "23 511"}, + // ⛔ These two sat in conformance.md and on the landing page with a stale + // figure while the tool reported four and five claims checked. The first + // says "total is", a word the alternation did not have; the second puts + // two words between "sum" and "is", which the three spelled-out + // determiners could not cross. Neither was ever a candidate, so both runs + // looked complete. + {"**The gate's current total is **23 511****", true, "23 511"}, + {"
  • CI gates file by file, not on a total: … The sum of those rows is 23,511, re-recorded at Ruby 4.0.9
    • ", true, "23,511"}, // Figure first: the landing page's style, in all three shapes it uses. {`
    23,511 ruby/spec language/+core/ examples passing`, true, "23,511"}, {`

    23,511 ruby/spec language/ + core/ examples pass, gated in CI file by file`, true, "23,511"},