Repository navigation
Conversation
|
@codex review |
Changelog🏠 Internal
Semantic Diff2 commits · Full page Count the build's allocations on generated projects in the benchmarksadfb074 · JonoPrest · Nov 09, 2026, 14:22 UTC
With - WideDeps: 240 modules that each use one or two values of four modules with 600 types and 1,200 values each. The edit adds a value to one of them, which compiles its users again. - Namespace: a package with Their counts are larger than all the others together, so the report adds them up on a line of their own, Locally, the signature copy pull requests take WideDeps clean from 505M to 333M words and its edit from 232M to 146M, and building the namespace map on demand takes Namespace clean from 611M to 203M; the other benchmarks' counts are unchanged. The builds of tests/tests now share countBuild and editModule with the generated projects. Time a chain of modules with the base's and the head's build systems511cf97 · JonoPrest · Nov 09, 2026, 14:27 UTC
Locally, with load around 5 to 9 on 8 cores, the ratio was 0.695 (0.646 to 0.825) for the pull request that starts dependents once a module's interface is final, 0.991 (0.948 to 1.055) for one that only changes namespaced packages, and 0.965 (0.921 to 1.078) with the same build system on both sides. The command took about 45 seconds. |
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Codex Review: Didn't find any major issues. Can't wait for the next one! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
✅ Compiler benchmarksAllocated wordsIn total: 683,548,494 → 683,548,310 (±0.00%). Generated projects: 1,333,102,129 → 1,333,102,112 (±0.00%). None of the 265 benchmarks changed by 0.1% or more. InstructionsIn total: 9,664,671,518 → 9,664,671,518 (±0.00%). None of the 33 benchmarks changed by 0.1% or more. Output sizeIn total: 25,569,598 → 25,569,598 (±0.00%). None of the 13 benchmarks changed by 0.1% or more. Wall time (indicative only)Clean builds of a generated chain of modules with the base's and the head's build systems in turn on this runner, on several domains. Wall time on a shared runner is noisy, so this never fails the job; it shows changes such as scheduling ones that the counts above can't. The ratio is the median of each round's head time over its base time, not the ratio of the two medians.
Pull request 511cf97 compared with base b82d944 in this run. |
rescript
@rescript/belt
@rescript/darwin-arm64
@rescript/linux-arm64
@rescript/linux-x64
@rescript/runtime
@rescript/win32-x64
commit: |
|
Developer playground preview: https://rescript-lang.github.io/rescript/dev-playground/?version=pr-9282 |
f5ff1ea to
511cf97
Compare
Why:
The benchmarks compile runtime and Belt files one at a time, whose
dependency graphs are small, and build tests/tests, whose modules use
a few small modules. Costs that grow with how many modules a build has
or loads barely show: a change that avoids copying the signatures of
large dependencies that a module hardly reads moved them by 0.4%, and
opening the map of a namespaced package, which is quadratic in its
modules per build, doesn't run at all.
With `--project`, `measure` now also builds two projects that the script
generates, clean and after an edit, on one worker domain and counting
allocated words like tests/tests:
- WideDeps: 240 modules that each use one or two values of four modules
with 600 types and 1,200 values each. The edit adds a value to one of
them, which compiles its users again.
- Namespace: a package with `"namespace": true` and 1,000 one-line
modules. The edit changes a module that nothing uses.
Their counts are larger than all the others together, so the report adds
them up on a line of their own, `Generated projects`, and `In total`
covers the other benchmarks as before. Each has its row and threshold like
any other benchmark. Every module of a project build loads the runtime's
interfaces, so the inputs of the generated projects and of tests/tests now
include a hash of the runtime sources: a pull request that changes the
runtime marks them as inputs changed rather than as regressions.
Locally, the signature copy pull requests take WideDeps clean from 505M
to 333M words and its edit from 232M to 146M, and building the namespace
map on demand takes Namespace clean from 611M to 203M; the other
benchmarks' counts are unchanged. The builds of tests/tests now share
countBuild and editModule with the generated projects.
Signed-off-by: Jono Prest <[email protected]>
Why:
A change to the build's scheduling, such as starting a module once the
interfaces it reads are final, does the same work sooner. Counting
allocated words and instructions can only show what such a change
costs, never what it saves.
`time <base checkout> <head checkout> <out.json>` builds a generated
project of 150 modules in a chain, each using the one before it, and a
module beside each, clean with the two build systems in turn on the
default number of worker domains: one build each that isn't counted,
then 10 rounds, the order alternating each round. `compare --timings`
adds a section to the report with the base's and the head's median times
and the median of the head's time over the base's in the same round, with
a 95% confidence interval from a percentile bootstrap of the rounds with a
fixed seed. The section never counts as a regression or an improvement;
when the times' file is missing or can't be read, it says they weren't
measured. In CI, the step runs after both measurements, each build may
take 2 minutes and the whole command 10, and a failure only leaves the
times out.
Locally, with load around 5 to 9 on 8 cores, the ratio was 0.695 (0.646
to 0.825) for the pull request that starts dependents once a module's
interface is final, 0.991 (0.948 to 1.055) for one that only changes
namespaced packages, and 0.965 (0.921 to 1.078) with the same build
system on both sides. The command took about 45 seconds.
Signed-off-by: Jono Prest <[email protected]>
511cf97 to
66b8aa3
Compare
Changelog🏠 Internal
Semantic Diff2 commits · Full page Count the build's allocations on generated projects in the benchmarksea9680b · JonoPrest · Nov 09, 2026, 14:22 UTC
With - WideDeps: 240 modules that each use one or two values of four modules with 600 types and 1,200 values each. The edit adds a value to one of them, which compiles its users again. - Namespace: a package with Their counts are larger than all the others together, so the report adds them up on a line of their own, Locally, the signature copy pull requests take WideDeps clean from 505M to 333M words and its edit from 232M to 146M, and building the namespace map on demand takes Namespace clean from 611M to 203M; the other benchmarks' counts are unchanged. The builds of tests/tests now share countBuild and editModule with the generated projects. Time a chain of modules with the base's and the head's build systems66b8aa3 · JonoPrest · Nov 09, 2026, 14:27 UTC
Locally, with load around 5 to 9 on 8 cores, the ratio was 0.695 (0.646 to 0.825) for the pull request that starts dependents once a module's interface is final, 0.991 (0.948 to 1.055) for one that only changes namespaced packages, and 0.965 (0.921 to 1.078) with the same build system on both sides. The command took about 45 seconds. |
✅ Compiler benchmarksAllocated wordsIn total: 683,549,129 → 683,549,928 (±0.00%). Generated projects: 1,333,104,744 → 1,333,105,108 (±0.00%). None of the 265 benchmarks changed by 0.1% or more. InstructionsIn total: 9,664,671,518 → 9,664,671,518 (±0.00%). None of the 33 benchmarks changed by 0.1% or more. Output sizeIn total: 25,569,598 → 25,569,598 (±0.00%). None of the 13 benchmarks changed by 0.1% or more. Wall time (indicative only)Clean builds of a generated chain of modules with the base's and the head's build systems in turn on this runner, on several domains. Wall time on a shared runner is noisy, so this never fails the job; it shows changes such as scheduling ones that the counts above can't. The ratio is the median of each round's head time over its base time, not the ratio of the two medians.
Pull request 66b8aa3 compared with base 70da3d1 in this run. |
|
Developer playground preview: https://rescript-lang.github.io/rescript/dev-playground/?version=pr-9282 |
The compiler benchmarks compile runtime and Belt files one at a time and build tests/tests, so costs that grow with the number of modules a build has or loads barely show: the signature copy pull requests (#9192, #9194) moved them by 0.4%, and nothing in them is namespaced. Scheduling changes such as #9191 do the same work sooner, which counting can only show as a cost.
This adds:
Two generated projects, built with
--projectlike tests/tests (one worker domain, allocated words, clean and after an edit):WideDeps: 240 modules using one or two values of four large modules.Namespace: a"namespace": truepackage of 1,000 one-line modules.Their counts are larger than all the others together, so the report adds them up on a line of their own,
Generated projects, andIn totalcovers the other benchmarks as before. Each still has its own row and the 1% threshold. The inputs of the project builds (these and tests/tests) now include a hash of the runtime sources, so a pull request that changes the runtime marks them as inputs changed rather than as regressions.time, which builds a generated chain of 150 modules (plus a module beside each) clean with the base's and the head's build systems in turn, 10 rounds, andcompare --timings, which reports both medians and the median of the per-round head/base ratios with a bootstrap 95% confidence interval. The section is marked indicative only and never counts as a regression. In CI, a failure or a run over 10 minutes only leaves the times out, and the report then says they weren't measured.Local results against master:
build/WideDeps/clean, #9194build/WideDeps/edit, #9194build/Namespace/clean, namespace map built on demandThe other benchmarks' counts are unchanged. The job takes about 25 s more for the generated projects and about 45 s for the timing, measured locally.
Open questions:
--roundschanges it.--project, as now, or have a flag of their own?Checklist
changes/README.md), or no release notes are needed and a maintainer should applychangelog:skip.CONTRIBUTING.md).git commit -s).