Repository navigation
Conversation
|
@codex review |
Changelog💅 Polish
Semantic Diff1 commit · Full page Look up the modules of a namespace by name instead of listing them all18dd323 · JonoPrest · Nov 09, 2026, 15:23 UTC
The components of a signature made only of aliases to persistent modules, such as a namespace's, are now looked up per name: Build modified (clean builds, default domains, medians of 15 interleaved pairs on a shared 8-core machine):
-4,900 modules, namespace wall 40.4 s user 169.6 s peak RSS 2,536 MB
- 480 modules, namespace wall 1.09 s user 3.45 s
-4,900 modules, none wall 6.03 s user 29.3 s
+4,900 modules, namespace wall 8.70 s user 38.7 s peak RSS 1,180 MB
+ 480 modules, namespace wall 1.02 s user 3.09 s
+4,900 modules, none wall 6.16 s user 29.8 s (no change: 95% CIs include 1)Every artifact of the 11 projects of a benchmark corpus (88,523 files, Review
|
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Codex Review: Didn't find any major issues. What shall we delve into next? Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #9287 +/- ##
=======================================
Coverage 81.92% 81.93%
=======================================
Files 456 456
Lines 65464 65515 +51
=======================================
+ Hits 53631 53679 +48
- Misses 11833 11836 +3
🚀 New features to boost your workflow:
|
✅ Compiler benchmarksAllocated wordsIn total: 683,548,708 → 683,547,921 (±0.00%). None of the 261 benchmarks changed by 0.1% or more. InstructionsIn total: 9,664,672,297 → 9,666,221,246 (+0.02%).
Output sizeIn total: 25,569,598 → 25,571,598 (+0.01%).
Pull request 18dd323 compared with base b82d944 in this run. |
Why:
Every module of a package with a namespace opens the namespace's
module, which maps each module of the package to its compiled name
(`module Price = Price-Shop`). Opening a module builds its components,
and building them walked every item of its signature, with a
substitution entry, two table insertions and an environment binding
per item, all kept for the rest of the compile. Each module of a
namespaced package therefore did work proportional to the size of the
package, and a clean build did quadratic work: on a package of about
4,900 modules, 46% of the build's CPU time went to building these
components.
The components of a signature made only of aliases to persistent modules,
such as a namespace's, are now looked up per name: `Tbl.of_source` makes a
table whose entries are computed when a name is first looked up or
visited, from an index of the signature's items. Folds and iterations
visit the names in the order of the table the eager loop built, which
decides, for example, the order of the candidates in a "did you mean"
hint. The entries are the ones the eager loop made: an alias to a
persistent module mentions no other item of its signature, so the
substitution and the environment that the loop extended item by item
can't change what it resolves to. Each entry still gets its own path and
a substitution with its own item, so the declarations it returns are
copies, shared as before in the `.cmi` and `.cmt` files. While the build
system keeps decoded interfaces across the compiles of a build, each
domain also keeps the indexes of the 16 most recently used such
interfaces, by physical signature, until the build's files change.
Build modified (clean builds, default domains, medians of 15 interleaved
pairs on a shared 8-core machine):
Before (text):
4,900 modules, namespace wall 40.4 s user 169.6 s peak RSS 2,536 MB
480 modules, namespace wall 1.09 s user 3.45 s
4,900 modules, none wall 6.03 s user 29.3 s
After (text):
4,900 modules, namespace wall 8.70 s user 38.7 s peak RSS 1,180 MB
480 modules, namespace wall 1.02 s user 3.09 s
4,900 modules, none wall 6.16 s user 29.8 s (no change: 95% CIs include 1)
Every artifact of the 11 projects of a benchmark corpus (88,523 files,
`.cmt` files included), of the runtime and of Belt is byte-identical to
master's, with one domain and with the default number.
Review: Do error messages and editor tools see the same modules in the
same order? Lookups, folds and iterations over the table return what the
eager table held, folds in the order of its keys. The new build test
checks the "did you mean" hint, which lists the namespace's modules in
that order, the type printed in an error, a module named like the
namespace and a local module that hides one of the package.
Review: How long are the indexes kept? Only while the build file store is
enabled, one per decoded interface: a domain keeps the 16 most recently used and drops
them when the store's generation changes, as it drops its decoded
interfaces. `bsc` and other compiles outside a build index the signature
on each compile.
Signed-off-by: Jono Prest <[email protected]>
87d7812 to
18dd323
Compare
rescript
@rescript/belt
@rescript/darwin-arm64
@rescript/linux-arm64
@rescript/linux-x64
@rescript/runtime
@rescript/win32-x64
commit: |
|
Developer playground preview: https://rescript-lang.github.io/rescript/dev-playground/?version=pr-9287 |
Every module of a package with a namespace opens the namespace's module, the map from each module of the package to its compiled name (
module Price = Price-Shop). Opening it built its components eagerly: for every alias, a substitution entry, two table insertions, a lazy value and an environment binding, all kept alive for the rest of the compile. That is work proportional to the size of the package for every module, so a clean build is quadratic in the number of modules. On a namespaced package of about 4,900 modules, a frame-pointer profile put 46% of the build's CPU samples underEnv.components_of_module_makerfor the namespace module.This change builds the components of a signature made only of aliases to persistent modules (a namespace map, and also modules such as Belt that only re-export units) on demand.
Tblgets a third kind of table,Tbl.of_source, whose entries are computed per name when they are looked up or visited, with folds and iterations in key order. The components index the signature once: a sorted array of names and a table from name to item and position. While the build file store is enabled, each domain also keeps the indexes of the 16 most recently used such interfaces, keyed by the physical signature thatBs_cmi_loadreturns to every compile of a build, and drops them when the store's generation changes, the same lifetime as the decoded interfaces.bscand compiles outside a build index the signature each time.Measured with clean builds on a shared 8-core Linux machine, 15 interleaved pairs, median B/A with bootstrap 95% CI:
Peak RSS of the large namespaced build drops from 2.5 GB to 1.2 GB at default domains.
scripts/compilerBenchmarks.jsis neutral (allocated words -0.01% in total, no benchmark changed by 0.1% or more; the output-size differences come from the two checkouts' path lengths).Output is unchanged. Every artifact of an 11-project benchmark corpus (88,523 files: JavaScript, GenType,
.cmi,.cmj,.ast,.cmt/.cmti, the namespaced packages and their dependencies), and of the runtime and Belt, is byte-identical to master's, with one domain and with the default number. A private corpus of three applications is identical as well (JavaScript,.cmi,.cmj,.astat both settings,.cmtat one domain; with default domains master's own.cmtfiles differ between runs, and the editor analysis andrescript-tools docoutputs over every source file are identical).The new build test
tests/build_tests/namespace_componentscovers lookups through the namespace, an alias and a localopenof a namespaced module, a module named like the namespace, a local module that hides a module of the package, a "did you mean" hint, whose candidates follow the order of the namespace's modules, and a type printed in an error. The hint fails when folds skip the lazy table or visit it unsorted.What reviewers may want to check:
is_alias_map: an alias to a persistent module mentions no other item, so leaving out the other items' substitution entries and the environment extended item by item can't change what a component resolves to.alias_map_components).interface_alias_map_index.Tbluser can see the new constructor:Tbl.tis abstract, and only the namespace components create one.addon such a table builds the whole tree (nothing adds to these tables today).Checklist
changes/README.md), or no release notes are needed and a maintainer should applychangelog:skip.CONTRIBUTING.md).git commit -s).