Skip to content

Key a build file by its own path when it is already normal - #9285

Open
JonoPrest wants to merge 1 commit into
masterfrom
jono/build-file-store-key-fast-path
Open

JonoPrest wants to merge 1 commit into
masterfrom
jono/build-file-store-key-fast-path

Conversation

@JonoPrest

Copy link
Copy Markdown
Contributor

Build_file_store.key normalized every path by splitting it into segments and joining them again, about 200 allocated words per call, although nearly every path the build looks up is already normal. A clean build of a 552-module React app calls it about 99k times. key now checks, without allocating, whether the path has no ., .. or empty segment and (on Windows) no backslash, and then returns it as is; other paths are normalized exactly as before. key is exposed so that a new ounit test can compare it with a copy of the previous normalization on edge cases (//, trailing /, dot segments anywhere, .../.a names, the empty path, backslashes, drive letters, UNC paths), on every path of up to 8 characters over / . \ a, and on 20,000 seeded random paths.

Measurement Change
Allocated words, builds of tests/tests (compilerBenchmarks.js) -2.27%
Allocated words, build/clean -2.48%
Allocated words, files compiled one by one 0.00%
552-module app, clean build, worker words (1 domain) -2.47%
552-module app, user time, 1 domain (40 pairs, 95% CI) -1.2% (-0.5% to -2.1%)
552-module app, user time, default domains (15 pairs, 95% CI) -2.0% (-1.0% to -2.8%)
552-module app, wall time no significant change

Build outputs (JS, .cmi, .cmj, .ast) are byte-identical on three real-world projects and their dependencies at default and one domain.

Checklist

  • Added a changelog fragment (see changes/README.md), or no release notes are needed and a maintainer should apply changelog:skip.
  • Added tests for the fix or feature.
  • Gave every commit a description below its subject, saying why it is made (see "Commit messages" in CONTRIBUTING.md).
  • Signed off every commit for the DCO (git commit -s).

@JonoPrest

Copy link
Copy Markdown
Contributor Author

@codex review

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Changelog

💅 Polish

  • Look up the files a build reads without normalizing paths that are already normal, so a clean build allocates about 2.5% less.

Semantic Diff

1 commit · Full page

Key a build file by its own path when it is already normal

51d5190 · JonoPrest · Nov 09, 2026, 14:38 UTC

Why

Build_file_store.key split every path into segments and joined them again, also when no segment was ., .. or empty and the result was the path itself. The store keys every read, so a request that looks up an interface along the load path built a list and a new string for each directory it tried, about 200 words a lookup.

key now first checks, without allocating, that the path has no ., .. or empty segment (other than the leading one of an absolute path) and no separator other than /, and then returns the path itself. Other paths are normalized as before, so every spelling of a file keeps the same entry.

key is now in the interface, so that a test can compare it with a copy of the previous normalization: on edge cases (//, trailing /, . and .. anywhere, ... and .a names, the empty path, backslashes, drive letters, UNC paths), on every path of up to 8 characters made of /, ., \ and a, and on 20,000 seeded random paths. The copy reads Sys.win32 like key, so the test holds on Windows too.

Measured with scripts/compilerBenchmarks.js against the previous commit: allocated words -2.27% in the builds of tests/tests (build/clean -2.48%), -0.78% in the generated projects, the files compiled one by one unchanged (the store is only used in a build). On a 552 module React app (clean build) the build allocates 2.47% fewer words at one domain. Its user time is 1.2% lower at one domain (40 alternating pairs, 95% CI 0.5% to 2.1%) and 2.0% lower at the default domains (15 pairs, 1.0% to 2.8%); its wall time is unchanged within the noise.

Review

  • Can a key change? No: a path that passes the check splits into segments that the normalization keeps in order, so joining them gives the path back; any other path takes the previous code.
  • What about Windows? There \ is also a separator, and a path with one takes the previous code; a path with only / separators, such as C:/a/b, is split the same way on both platforms.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-09T15:55:04.030061Z c1b4ff3 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Hooray!

Reviewed commit: c1b4ff3f1f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

🎉 Compiler benchmarks

Allocated words

In total: 683,552,746 → 679,251,058 (-0.63%).

🎉 Improvement: 2 benchmarks allocate more than 1% less.

Benchmark Base Head Change
build/clean 246,759,391 243,416,916 -1.35%
build/shared-edit 75,191,255 74,405,644 -1.04%
build/no-op 9,848,483 9,763,862 -0.86%
build/leaf-edit 13,851,969 13,762,988 -0.64%

Instructions

In total: 9,664,672,297 → 9,664,675,329 (±0.00%).

None of the 33 benchmarks changed by 0.1% or more.

Output size

In total: 25,569,598 → 25,569,587 (±0.00%).

None of the 13 benchmarks changed by 0.1% or more.

Pull request 51d5190 compared with base b82d944 in this run.

@codecov

codecov Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.33333% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.93%. Comparing base (70da3d1) to head (dc61691).

Files with missing lines Patch % Lines
compiler/ext/build_file_store.ml 90.90% 3 Missing ⚠️
tests/ounit_tests/ounit_build_file_store_tests.ml 95.23% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #9285      +/-   ##
==========================================
+ Coverage   81.92%   81.93%   +0.01%     
==========================================
  Files         456      456              
  Lines       65464    65522      +58     
==========================================
+ Hits        53631    53687      +56     
- Misses      11833    11835       +2     
Files with missing lines Coverage Δ
tests/ounit_tests/ounit_build_file_store_tests.ml 96.38% <95.23%> (-0.17%) ⬇️
compiler/ext/build_file_store.ml 91.33% <90.90%> (+1.24%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@pkg-pr-new

pkg-pr-new Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

rescript

npm i https://pkg.pr.new/rescript@9285

@rescript/belt

npm i https://pkg.pr.new/@rescript/belt@9285

@rescript/darwin-arm64

npm i https://pkg.pr.new/@rescript/darwin-arm64@9285

@rescript/linux-arm64

npm i https://pkg.pr.new/@rescript/linux-arm64@9285

@rescript/linux-x64

npm i https://pkg.pr.new/@rescript/linux-x64@9285

@rescript/runtime

npm i https://pkg.pr.new/@rescript/runtime@9285

@rescript/win32-x64

npm i https://pkg.pr.new/@rescript/win32-x64@9285

commit: dc61691

@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown

@JonoPrest
JonoPrest marked this pull request as ready for review November 9, 2026 16:08
@JonoPrest
JonoPrest requested a review from cknitt November 9, 2026 16:08
@JonoPrest
JonoPrest force-pushed the jono/build-file-store-key-fast-path branch from c1b4ff3 to 51d5190 Compare November 9, 2026 16:48
Why:
    `Build_file_store.key` split every path into segments and joined them
    again, also when no segment was `.`, `..` or empty and the result was
    the path itself. The store keys every read, so a request that looks
    up an interface along the load path built a list and a new string for
    each directory it tried, about 200 words a lookup.

`key` now first checks, without allocating, that the path has no `.`,
`..` or empty segment (other than the leading one of an absolute path)
and no separator other than `/`, and then returns the path itself. Other
paths are normalized as before, so every spelling of a file keeps the
same entry.

`key` is now in the interface, so that a test can compare it with a copy
of the previous normalization: on edge cases (`//`, trailing `/`, `.`
and `..` anywhere, `...` and `.a` names, the empty path, backslashes,
drive letters, UNC paths), on every path of up to 8 characters made of
`/`, `.`, `\` and `a`, and on 20,000 seeded random paths. The copy reads
`Sys.win32` like `key`, so the test holds on Windows too.

Measured with scripts/compilerBenchmarks.js against the previous commit:
allocated words -2.27% in the builds of tests/tests (build/clean
-2.48%), -0.78% in the generated projects, the files compiled one by one
unchanged (the store is only used in a build). On a 552 module React app
(clean build) the build allocates 2.47% fewer words at one domain. Its
user time is 1.2% lower at one domain (40 alternating pairs, 95% CI 0.5%
to 2.1%) and 2.0% lower at the default domains (15 pairs, 1.0% to 2.8%);
its wall time is unchanged within the noise.

Review: Can a key change? No: a path that passes the check splits into
    segments that the normalization keeps in order, so joining them gives
    the path back; any other path takes the previous code.
Review: What about Windows? There `\` is also a separator, and a path
    with one takes the previous code; a path with only `/` separators,
    such as `C:/a/b`, is split the same way on both platforms.

Signed-off-by: Jono Prest <[email protected]>
@JonoPrest
JonoPrest force-pushed the jono/build-file-store-key-fast-path branch from 51d5190 to dc61691 Compare November 9, 2026 22:04
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown

Changelog

💅 Polish

  • Look up the files a build reads without normalizing paths that are already normal, so a clean build allocates about 2.5% less.

Semantic Diff

1 commit · Full page

Key a build file by its own path when it is already normal

dc61691 · JonoPrest · Nov 09, 2026, 14:38 UTC

Why

Build_file_store.key split every path into segments and joined them again, also when no segment was ., .. or empty and the result was the path itself. The store keys every read, so a request that looks up an interface along the load path built a list and a new string for each directory it tried, about 200 words a lookup.

key now first checks, without allocating, that the path has no ., .. or empty segment (other than the leading one of an absolute path) and no separator other than /, and then returns the path itself. Other paths are normalized as before, so every spelling of a file keeps the same entry.

key is now in the interface, so that a test can compare it with a copy of the previous normalization: on edge cases (//, trailing /, . and .. anywhere, ... and .a names, the empty path, backslashes, drive letters, UNC paths), on every path of up to 8 characters made of /, ., \ and a, and on 20,000 seeded random paths. The copy reads Sys.win32 like key, so the test holds on Windows too.

Measured with scripts/compilerBenchmarks.js against the previous commit: allocated words -2.27% in the builds of tests/tests (build/clean -2.48%), -0.78% in the generated projects, the files compiled one by one unchanged (the store is only used in a build). On a 552 module React app (clean build) the build allocates 2.47% fewer words at one domain. Its user time is 1.2% lower at one domain (40 alternating pairs, 95% CI 0.5% to 2.1%) and 2.0% lower at the default domains (15 pairs, 1.0% to 2.8%); its wall time is unchanged within the noise.

Review

  • Can a key change? No: a path that passes the check splits into segments that the normalization keeps in order, so joining them gives the path back; any other path takes the previous code.
  • What about Windows? There \ is also a separator, and a path with one takes the previous code; a path with only / separators, such as C:/a/b, is split the same way on both platforms.

@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown

🎉 Compiler benchmarks

Allocated words

In total: 683,552,616 → 679,250,674 (-0.63%).

🎉 Improvement: 2 benchmarks allocate more than 1% less.

Benchmark Base Head Change
build/clean 246,759,218 243,416,682 -1.35%
build/shared-edit 75,191,224 74,405,570 -1.04%
build/no-op 9,848,483 9,763,825 -0.86%
build/leaf-edit 13,852,043 13,762,949 -0.64%

Instructions

In total: 9,664,672,297 → 9,664,675,329 (±0.00%).

None of the 33 benchmarks changed by 0.1% or more.

Output size

In total: 25,569,598 → 25,569,587 (±0.00%).

None of the 13 benchmarks changed by 0.1% or more.

Pull request dc61691 compared with base 70da3d1 in this run.

@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant