Repository navigation
Conversation
|
@codex review |
The entry names #9191, so the changelog points at this PR.
Changelog💅 Polish
Semantic Diff3 commits · Full page Let a build system start a request before its dependencies publish924d761 · JonoPrest · Nov 09, 2026, 08:02 UTC
A request now takes hooks from the build system. interface_computed receives the contents of the .cmi an implementation is about to write, once no warning reported as an error is pending; the contents are computed once (Cmi_format.cmi_contents) and also written. redirect_cmi hands Bs_cmi_load the contents of a .cmi that is not published yet, which it reads instead of the build file store; the store never sees them. before_effect is called before the request writes a .cmi, .cmt, .cmti, binary AST or a PPX's input, runs a command such as a PPX, or takes a command line action that writes a file (-e, -format); a build system that started the request early waits there, or raises to abandon it before any effect. Build_file_store.key is exported so that a build system matches paths as the store does. bsc attaches no hooks and behaves as before. Review
Start dependents once a module's interface is final63af9ae · JonoPrest · Nov 09, 2026, 08:02 UTC
When a slot would otherwise idle, a module can now start as soon as each unfinished dependency's interface is final: an implementation without .resi has computed the .cmi it is about to write, or an interface file has compiled. The early request reads those .cmi files from memory and waits at a gate before writing a file, even one the build holds in memory until publishing, running a command or returning its result. The scheduler adopts it at the point it would have started it (dependencies complete, module still dirty), so its code generation sees the dependencies' .cmj files, and artifacts, diagnostics and compiler.log are unchanged. Otherwise it is abandoned without effects, together with anything that started on top of it. Redirected paths are matched as the build file store keys them. The directories in which the build publishes interfaces are spelled from canonical package roots, both in the redirected paths and on the load path, so a lookup there that matches no redirected path names another file. A lookup of a redirected file name in any other directory, which may be an alias from user flags (with -nostdlib those come first), or one that differs only in case, waits for adoption instead. Whether an announced interface changes the published one is read from disk, so that an abandoned request never fills the store. Modules with js-post-build commands or lowercase .cmi names don't release their dependents early, and with one domain nothing changes. Review
✨ Build a .cmi's contents only when something reads themee9b9b2 · JonoPrest · Nov 09, 2026, 13:36 UTC · new
Review
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Codex Review: Didn't find any major issues. What shall we delve into next? Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
The entry names #9191, so the changelog points at this PR. Signed-off-by: Jono Prest <[email protected]>
d681ed4 to
93e6375
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #9191 +/- ##
==========================================
+ Coverage 81.92% 81.96% +0.03%
==========================================
Files 456 456
Lines 65469 65640 +171
==========================================
+ Hits 53635 53800 +165
- Misses 11834 11840 +6
🚀 New features to boost your workflow:
|
rescript
@rescript/belt
@rescript/darwin-arm64
@rescript/linux-arm64
@rescript/linux-x64
@rescript/runtime
@rescript/win32-x64
commit: |
|
Developer playground preview: https://rescript-lang.github.io/rescript/dev-playground/?version=pr-9191 |
|
@codex review |
|
Codex Review: Didn't find any major issues. Hooray! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
04dca39 to
7bcd9d0
Compare
|
@codex review |
7bcd9d0 to
8a61aaa
Compare
|
@codex review |
|
Codex Review: Didn't find any major issues. You're on a roll. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
8a61aaa to
4203498
Compare
✅ Compiler benchmarksAllocated wordsIn total: 685,973,969 → 686,132,419 (+0.02%). None of the 261 benchmarks changed by 0.1% or more. InstructionsIn total: 9,675,830,335 → 9,676,274,684 (±0.00%). None of the 33 benchmarks changed by 0.1% or more. Output sizeIn total: 28,859,904 → 28,860,390 (±0.00%). None of the 13 benchmarks changed by 0.1% or more. Pull request ee9b9b2 compared with base e941d27 in this run. |
7e6f356 to
b6806c0
Compare
Why:
A dependent only needs a module's .cmi to type check, but a build
system that runs requests in process can only start it after that
module has also produced its lambda, JavaScript, .cmj and .cmt.
A request now takes hooks from the build system. interface_computed
receives the contents of the .cmi an implementation is about to write,
once no warning reported as an error is pending; the contents are
computed once (Cmi_format.cmi_contents) and also written. redirect_cmi
hands Bs_cmi_load the contents of a .cmi that is not published yet,
which it reads instead of the build file store; the store never sees
them. before_effect is called before the request writes a .cmi, .cmt,
.cmti, binary AST or a PPX's input, runs a command such as a PPX, or
takes a command line action that writes a file (-e, -format); a build
system that started the request early waits there, or raises to abandon
it before any effect. Build_file_store.key is exported so that a build
system matches paths as the store does. bsc attaches no hooks and
behaves as before.
Review: Can a redirected interface reach a later request through the
store or the decoded interface cache? Only as the same bytes. The
dependency writes the very string it announced through the store, which
holds it until the build publishes it, so a domain's cache may reuse an
interface it decoded from the redirect for the published file. A request
that is abandoned or fails drops its domain's cache.
Review: Why gate writes that the build file store already holds? The
store is shared by the requests of a build, and a request that is
abandoned may still run beside the request that replaces it, so a held
.cmi or .cmt it wrote could replace its successor's before publication.
Signed-off-by: Jono Prest <[email protected]>
Why:
On builds limited by a long dependency chain most job slots sit idle
on the critical path: a 136 module library gets only 1.6x from 7
domains.
When a slot would otherwise idle, a module can now start as soon as each
unfinished dependency's interface is final: an implementation without
.resi has computed the .cmi it is about to write, or an interface file
has compiled. The early request reads those .cmi files from memory and
waits at a gate before writing a file, even one the build holds in
memory until publishing, running a command or returning its result. The
scheduler adopts it at the point it would have started it (dependencies
complete, module still dirty), so its code generation sees the
dependencies' .cmj files, and artifacts, diagnostics and compiler.log
are unchanged. Otherwise it is abandoned without effects, together with
anything that started on top of it.
Redirected paths are matched as the build file store keys them. The
directories in which the build publishes interfaces are spelled from
canonical package roots, both in the redirected paths and on the load
path, so a lookup there that matches no redirected path names another
file. A lookup of a redirected file name in any other directory, which
may be an alias from user flags (with -nostdlib those come first), or
one that differs only in case, waits for adoption instead. Whether an
announced interface changes the published one is read from disk, so
that an abandoned request never fills the store. Modules with
js-post-build commands or lowercase .cmi names don't release their
dependents early, and with one domain nothing changes.
Review: Can a dependent see an interface that differs from master's?
It reads the exact contents the dependency then writes, and it is only
adopted after the dependency completed. If the dependency fails later,
for example in code generation, the dependent is abandoned before it
wrote, ran or reported anything.
Signed-off-by: Jono Prest <[email protected]>
b6806c0 to
63af9ae
Compare
Why:
Since the interface is announced before it is written, every .cmi
write built the whole file (the signature plus the marshalled CRCs and
flags), even with no build system listening and when `check_exists`
then found the same file on disk and wrote nothing. That is about
5,000 more words per runtime module in the compiler benchmarks, over
1% for the typed array modules.
`interface_computed` is now optional in `Compiler_request_state.hooks`,
and rewatch passes it only when a dependent may start early (as
`on_interface_computed` already was). Without it, `create_cmi` digests
the signature as before and builds the rest of the file only when it
writes it, as on master.
Review: Can a .cmi or a speculation change? No: the bytes are built by
the same functions in the same order, and a listener receives them
exactly as before. Only when nobody listens is the concatenation
skipped, and then only when the file is not written.
Signed-off-by: Jono Prest <[email protected]>
A dependent only needs a module's
.cmito type check, but OCaml rewatch starts it only after that module has also produced its lambda, JavaScript,.cmjand.cmt. On builds limited by a long dependency chain most job slots sit idle on the critical path.Rebased on the flat
lib/rescriptlayout (#9245).How it works
.resihas computed the.cmiit is about to write (and has no pending warning as error), or.cmicontents from memory (Compiler_request_state.redirect_cmi, checked inBs_cmi_loadbefore the build file store) and waits at a gate before it writes any file, runs any command (PPX) or returns a result..cmjfiles, so artifacts, diagnostics andcompiler.logare the same as on master.Held writes
The build file store now holds ASTs, interfaces and typed trees until their request succeeded. That doesn't replace the gate:
.cmjfiles, which aren't held and exist only once a dependency completed..cmior partial.cmtit wrote could replace its successor's before publication.The redirected path is the file the dependency writes and the store holds, so both serve the same string and the decoded interface cache stays valid after publication. Dating back the AST of a failed compile and publishing its partial typed tree happen only after adoption.
Redirected paths are matched as the store keys them (
Build_file_store.key). Each package'slib/rescriptis spelled from its canonical package root, on the load path too (-I .from the request's directory), so a lookup there that matches no redirected path is another file. A lookup of a redirected name anywhere else (an alias from user-Iflags), or one that differs only in case, waits for adoption instead of reading the disk. Whether an announced interface changes the published one is read from disk, so an abandoned request never fills the store. Modules withjs-post-buildcommands or lowercase.cminames don't release their dependents early. With one domain nothing changes.Performance
Clean builds, OCaml 5.5.2, 8 vCPU Linux VM, interleaved pairs against master. Median B/A with bootstrap 95% CI, a library of 132 modules with a deep dependency chain:
Projects bound by total CPU (a 552 module React app, a 391 module backend service) showed no change before the rebase.
Testing
.cmi,.cmjand.astare byte identical to master on three production codebases at default and 1 domain..cmtis byte identical at 1 domain and semantically equivalent at default (master's own.cmtbytes vary between builds on OCaml 5.5).Process(gate, adopt, abandon, failed dependency, interrupt), the compiler hooks (gate before every file and command, including-bs-ast -o,-e,-format, PPXs), redirects vs the store and its interface cache, scheduler eligibility and exclusions, redirect matching inlib/rescript, andcheck_pipelining.sh, which compares 1 and 4 domain builds byte for byte and checks speculation happened.REWATCH_SPECULATION_LOG(test only) records start, adopt and abandon.make test,make test-rewatch, the rewatch unit tests andrewatch-ocaml/tests/run.shpass.