Skip to content

CUDA backend: move merge_ptes into backends/cuda/scripts - #25323

Draft
Gasoonjia wants to merge 1 commit into
mainfrom
cuda-move-merge-ptes-scripts
Draft

Gasoonjia wants to merge 1 commit into
mainfrom
cuda-move-merge-ptes-scripts

Conversation

@Gasoonjia

@Gasoonjia Gasoonjia commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #25315 — review/merge that first; this PR's diff is against its branch.

What

Moves the other standalone CUDA backend tool into the scripts/ package #25315 starts: backends/cuda/merge_ptes.py → backends/cuda/scripts/merge_ptes.py, shown by Git as a rename. Every reference — the CUDA CI workflow's merge step, the documented command in examples/cuda/README.md, the imports and patch targets in test_merge_ptes.py (now also registered as a Buck python_unittest), and the :merge_ptes Buck target — now uses executorch.backends.cuda.scripts.merge_ptes.

The old module path is retired outright, with no forwarding shim — a deliberate maintainer call for this move: merge_ptes is a standalone, one-shot tool (run it once to produce a file), not a library surface other code builds on, every in-repo caller is updated here, and keeping a forwarding module at the old path would defeat the move (the PR stops reading as a rename, and the old path lingers as a second home for the tool). The release notes: devtools label calls the path change out (the repo has no breaking-change label to add, though release.yml reserves a category for one).

The content changes in the moved file are the output-writing changes below plus three wrapped overlong lines. Both outputs write through the shared scripts/_atomic_output writer from the base PR's :scripts_atomic_output library. The PTD is copied first with FileBackedData (8 MiB chunks, as fast as the old shutil.copyfile path), so a failed copy leaves the existing output PTE untouched instead of pairing a new PTE with a stale weights blob. Both outputs keep their inputs' permissions and remove their temp files when a write fails, instead of leaving 0600 or truncated files behind; an existing --output-ptd symlink is now replaced by a regular file rather than written through. The --output-ptd requires --input-ptd check now runs with the other argument checks, before any output is written.

Tests

test_merge_ptes.py import strings point at the new path; the CLI test now also asserts the merged PTE and PTD keep distinctive input modes (0o640/0o604) with no temp files left, --output-ptd without --input-ptd is rejected before the merge runs and writes nothing, and a failed PTD copy (patched FileBackedData) leaves pre-existing outputs byte-identical — the last also covers the PTD-first ordering. The suite needs flatc, unavailable in my local environment, and runs in CI (now also as a Buck python_unittest); the argument-check test passes locally. The 22 test_remap_pte_devices.py tests still pass on this stack.

@Gasoonjia Gasoonjia added the release notes: devtools Changes to dev tooling, for example the debugger & profiler label Nov 20, 2026
@pytorch-bot

pytorch-bot Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/25323

Note: Links to docs will display an error until the docs builds have been completed.

⏳ 45 Pending, 1 Unrelated Failure

As of commit f7cff09 with merge base 21c7c99 (image):

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 20, 2026
@Gasoonjia

Copy link
Copy Markdown
Contributor Author

@claude /pr-review

@claude

claude Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

Claude finished @Gasoonjia's task in 11m 58s —— View job


PR Review: #25323

Needs discussion: the move deletes executorch.backends.cuda.merge_ptes outright, but docs/source/api-life-cycle.md requires a deprecation period before removing a non-experimental API. Three smaller issues are in the details: the .ptd is still written non-atomically, the shared helper is listed in two Buck targets, and the new permission behavior has no test.

Detailed review

Summary

The PR moves backends/cuda/merge_ptes.py to backends/cuda/scripts/merge_ptes.py, updates its references, and writes the PTE output through the _atomic_output writer from #25315. Problems found:

  • Removing the old module path outright may conflict with the deprecation policy.
  • The atomic write covers only one of the tool's two outputs.
  • The shared helper files now belong to two Buck targets.
  • The permission change has no test.

Code Quality

  • Only the .pte output goes through the atomic writer (backends/cuda/scripts/merge_ptes.py:723-728).
    • The .ptd is the weights blob and usually much larger than the PTE. It is still written in place with shutil.copyfile, after model.pte has already been replaced.
    • If that copy is interrupted (ENOSPC, Ctrl-C, CI timeout), a new PTE is left next to a truncated aoti_cuda_blob.ptd. _atomic_output.py:9-13 says that state must never be visible.
    • The two outputs of one run can also end up with different modes. The PTE gets the mode of sources[0]; the PTD gets the umask default, or keeps the existing file's mode.
    • So the description's "writes its output through the shared writer" is true only for the PTE.
    • In the same block, and already present before this PR: the --output-ptd requires --input-ptd check (lines 724-725) runs after the PTE write. That invalid invocation replaces the output PTE and then raises.
    • Fix: move that check up with the argument checks at lines 697-702. Write the PTD via write_atomically(args.output_ptd, lambda out: shutil.copyfileobj(src, out), mode_from=args.input_ptd[0]), with src opened from args.input_ptd[0], and keep the same-path guard. Fix this →

Infrastructure

  • Two Buck targets own the shared helper sources (backends/cuda/BUCK:155-156).
    • scripts/__init__.py and scripts/_atomic_output.py are now in the srcs of both :merge_ptes and :remap_pte_devices (lines 173-174).
    • Every other source in this file has exactly one owner, and shared modules get their own library. For example, :cpu_count (lines 6-16) is a dep of :triton_kernels (line 231).
    • Any binary or test that depends on both tools gets the same modules from two libraries, and an owner lookup for _atomic_output.py returns two targets. I could not run Buck here, so it is unverified whether Buck2 actually fails on this.
    • Fix: give the two files their own python_library (e.g. :scripts_atomic_output, visibility //executorch/backends/cuda/...) and add it to the deps of both tools. Fix this →

Testing

  • The new output mode is untested for merge (backends/cuda/tests/test_merge_ptes.py:633).
    • This PR changes the merged PTE's mode from 0600 to the first input's mode. test_cli_writes_merged_pte_and_reuses_base_ptd still checks only output_pte.is_file().
    • The sibling remap test does check the mode (test_remap_pte_devices.py:324-328).
    • Dropping mode_from, or pointing it at a different input, would still pass. The PR also says this suite was not run locally.
    • Fix: before merge_main(), chmod first.pte_path to a distinctive mode such as 0o640. Then assert that the output's st_mode & 0o777 matches, and that output/ contains only model.pte and aoti_cuda_blob.ptd, with no leftover .model.pte.* temp file. Fix this →

Backward Compatibility

  • The old module path is removed with no deprecation period (backends/cuda/merge_ptes.py, deleted).
    • These now fail with ModuleNotFoundError:
      • python -m executorch.backends.cuda.merge_ptes ..., the command examples/cuda/README.md:46 documented until this PR
      • imports of CudaPteInput, merge_cuda_pte_files and merge_cuda_pte_files_with_provenance
    • Nothing marks this tool Experimental. The README's only experimental marker (line 7) is for ROCm. So under docs/source/api-life-cycle.md:20-31 it is Stable, and must go through Deprecated before it is removed.
    • Internal Buck users also break, and without warning. The PUBLIC :merge_ptes target (backends/cuda/BUCK:153) keeps its name but no longer provides the old module, so dependents still build and fail only at import time.
    • Fix: keep backends/cuda/merge_ptes.py as a shim until the deprecation period ends. It should re-export from executorch.backends.cuda.scripts.merge_ptes, emit a FutureWarning, run main() under __main__, and stay in the :merge_ptes srcs. Fix this →
    • Low confidence on severity: the checkout is shallow and has no tags, so I couldn't check whether the old path shipped in a tagged release. If it only ever existed on main, removing it outright is likely fine, and only the label point below applies.
    • The description says the release notes: devtools label covers the move. That label is for developer tools such as the debugger and profiler (CONTRIBUTING.md:421). Backend changes use Release notes: <backend> (CONTRIBUTING.md:419), so with the current label the removal won't appear in the CUDA section of the release notes.

Recommendation

Needs Discussion: if executorch.backends.cuda.merge_ptes shipped in a release, this needs a deprecation shim (blocking); if not, it can merge without one. The other three items are minor and can be fixed in this PR.


  • Gather context (diff vs origin/cuda-remap-pte-devices, CLAUDE.md, checklist)
  • Run independent reviews (3 reviewers)
  • Consolidate findings
  • Fact-check findings
  • Post final review

Three independent reviewers covered all five changed files, and I re-read the cited code for each finding above. I couldn't check release tags or run Buck from this job. I didn't run the test suites either; the merge suite needs flatc.

@Gasoonjia

Copy link
Copy Markdown
Contributor Author

@claude addressed, with one deliberate exception:

  • PTD write + late check: the .ptd now goes through write_atomically too (via shutil.copyfileobj, keeping the same-path guard and the input's mode), and the --output-ptd requires --input-ptd check moved up with the other argument checks, before any output is written.
  • Shared helper ownership: scripts/__init__.py and scripts/_atomic_output.py moved into their own :scripts_atomic_output library (following the :cpu_count pattern); both tools depend on it, and neither lists the files in its own srcs anymore.
  • Mode test: test_cli_writes_merged_pte_and_reuses_base_ptd now chmods the first input to 0o640 and asserts the merged PTE has exactly that mode and the output directory contains only model.pte and aoti_cuda_blob.ptd.

Kept as-is, deliberately: retiring executorch.backends.cuda.merge_ptes without a deprecation shim. This is the maintainer's call for this move — merge_ptes is a standalone, one-shot tool (run it once to produce a file), not a library surface other code builds on, and every in-repo caller was updated in this PR. Keeping a forwarding module also defeats the point of the move (the PR stops reading as a rename, and the old path lingers as a second home for the tool). The release notes: devtools label calls the path change out.

@Gasoonjia

Copy link
Copy Markdown
Contributor Author

@claude /pr-review

@claude

claude Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

Claude finished @Gasoonjia's task in 40s —— View job


Follow-up review: #25323

All three fixes are on the branch and do what you described. I'm not raising the deprecation shim again, since you've made that call. Nothing blocking is left from my side.

Details

Fixes I checked

  • PTD write and the moved check
    • --output-ptd requires --input-ptd now runs with the other argument checks (backends/cuda/scripts/merge_ptes.py:701-702), before anything is written.
    • The .ptd is now written through write_atomically with shutil.copyfileobj and mode_from=args.input_ptd[0] (lines 727-733). The same-path guard is still there.
    • The source file is opened outside the writer, so a failed open writes nothing and leaves no temp file behind.
  • Buck ownership
    • :scripts_atomic_output (backends/cuda/BUCK:151-161) is now the only target that lists scripts/__init__.py and scripts/_atomic_output.py.
    • Both :merge_ptes and :remap_pte_devices depend on it, and its visibility is limited to //executorch/backends/cuda/.... This follows the :cpu_count pattern.
  • Mode test
    • test_cli_writes_merged_pte_and_reuses_base_ptd sets the first input to 0o640 before merge_main().
    • It asserts that the merged PTE has exactly that mode and that output/ contains only model.pte and aoti_cuda_blob.ptd. That would catch a dropped or wrong mode_from and a leftover .model.pte.* temp file.

Deprecation shim

I accept your decision to remove the old module path without a shim. Every in-repo caller is updated, and I couldn't confirm the old path ever shipped in a release.

Optional notes

  • The two outputs are not replaced together. Each file is now atomic on its own, but the PTE is replaced before the PTD copy starts. If the PTD copy fails, the new model.pte sits next to the previous aoti_cuda_blob.ptd, if there was one. Neither file is truncated, so this is much better than before. Fixing it fully would mean staging both files and then replacing both, which is more than this PR needs.
  • The release-notes label. CONTRIBUTING.md:419-421 maps backend changes to Release notes: <backend>, so the path change may not show up under CUDA in the release notes. If that's fine with you, there's nothing to change.

Verification

I read the code and the diff against origin/cuda-remap-pte-devices. I did not run test_merge_ptes.py because it needs flatc, and I did not run Buck. CI will cover both.

  • Confirm the fixes are on the branch
  • Check the PTD atomic write and the earlier argument check
  • Check the :scripts_atomic_output Buck library
  • Check the mode test
  • Reply on the shim decision

@claude

claude Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

Claude finished @Gasoonjia's task in 16m 24s —— View job


PR Review: #25323

Approve: nothing blocks, though the merged PTE is still replaced before the PTD copy starts, this round's PTD and argument-check changes have no tests, and the removal isn't labeled breaking-change.

Detailed review

Summary

The PR moves merge_ptes into backends/cuda/scripts/, points every reference at the new module, and now writes both outputs through write_atomically. Nothing blocks; the three problems below are minor.

Code Quality

  • The merged PTE is still replaced before the PTD copy starts (backends/cuda/scripts/merge_ptes.py:721-733).
    • Each file is now atomic, but the pair isn't. model.pte is renamed into place (_atomic_output.py:45) before the PTD copy begins. If the copy then fails (ENOSPC, Ctrl-C, an unwritable --output-ptd directory), the run exits non-zero and leaves the new PTE next to the previous run's PTD, or next to no PTD.
    • The atomic copy makes ENOSPC more likely when re-running into an existing output directory. Until the rename, the temp file needs room for a second full PTD; the old in-place copyfile didn't.
    • Before this PR, the same failure left a visibly truncated PTD. Now the old PTD stays intact, so the mismatched pair no longer looks broken.
    • This is the part of last round's PTD point that my suggested fix didn't cover.
    • Fix: move the PTD block (lines 725-733) to between the merge (line 720) and the PTE write. The large copy then finishes before any output is replaced, and only the small in-memory PTE write comes after the first replace. The reorder is safe because the merged PTE takes no data from the PTDs, and the merge has finished hashing them by line 720. Fix this →

Testing

  • This round's PTD-side changes are untested (backends/cuda/tests/test_merge_ptes.py:606-639).
    • Only first.pte_path is chmodded, and only the PTE's mode is checked. Dropping mode_from=args.input_ptd[0] (merge_ptes.py:732) leaves the PTD at the temp file's 0600, and the suite still passes.

    • Nothing exercises the moved --output-ptd requires --input-ptd check (merge_ptes.py:701-702), so moving it back below the PTE write would go unnoticed.

    • The directory-listing assertion (lines 636-639), which I suggested last round, runs only on the success path, where the old code also left no temp file. Neither suite makes a write fail, so the claim that the tool "removes its temp file when a write fails" has no test.

    • Fix:

      • chmod first.ptd_path to a mode distinct from the PTE's (e.g. 0o604) and assert that output_ptd has it;
      • add a test that passes --output-ptd without --input-ptd, expects the ValueError, and asserts that output_pte doesn't exist (like test_cli_dry_run_writes_nothing at test_remap_pte_devices.py:284);
      • add one that makes the PTD copy raise (e.g. patch shutil.copyfileobj with side_effect=OSError) and asserts that no .aoti_cuda_blob.ptd.* file is left. With the reorder above, it can also assert that model.pte wasn't written.

      Fix this →

Backward Compatibility

  • The removal isn't filed as a breaking change (PR labels; .github/release.yml:8-11).
    • The generated release notes put a PR under "Breaking Changes 🛠" only if it has the breaking-change or Semver-Major label. release notes: devtools puts it under Devtools (lines 60-63). Users whose python -m executorch.backends.cuda.merge_ptes stops working won't find it among the breaking changes.
    • If the old path shipped in a release, add breaking-change. I couldn't check whether it did: the old path is on main, but this checkout is shallow and has no tags.
    • Correction to my last review: release.yml has no CUDA section, so Devtools is a reasonable category. Only the breaking-change listing is missing.

Recommendation

Approve: nothing blocks. The reorder and the tests are small enough to land in this PR. The label only matters if the old path was in a release.


  • Gather context (diff vs origin/cuda-remap-pte-devices, CLAUDE.md, checklist)
  • Run independent reviews (3 reviewers)
  • Consolidate findings
  • Fact-check findings
  • Post final review

Three independent reviewers covered all five changed files, and I re-read the cited code for each finding. I couldn't run Python or Buck in this job, so no tests were run (the merge suite also needs flatc). I also couldn't check release tags.

@Gasoonjia

Copy link
Copy Markdown
Contributor Author

@claude /pr-review

@claude

claude Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

Claude finished @Gasoonjia's task in 17m 0s —— View job


PR Review: #25323

Approve: nothing blocks merging. The details list five minor items, mainly two behavior changes with no test (the PTD's new mode and the earlier --output-ptd check) and a release-notes label that doesn't mark the retired module path as a breaking change.

Detailed review

Summary

The PR moves merge_ptes to backends/cuda/scripts/, updates every reference to it, and writes both outputs through write_atomically. All five problems are minor:

  • two changed behaviors have no test;
  • the PTE is still replaced before the PTD;
  • a new comment goes against CLAUDE.md;
  • the PR description is out of date;
  • the release-notes label puts the removed module path outside Breaking Changes.

Code Quality

  • The PTE is still replaced before the PTD is copied (backends/cuda/scripts/merge_ptes.py:721-733).
    • This order predates the PR. Each file is now written atomically, but the two files are not replaced as a pair.
    • Trigger: re-running into an existing output directory when the multi-GB PTD copy fails, for example with ENOSPC. The new model.pte is then already in place next to the previous aoti_cuda_blob.ptd, or next to no PTD at all.
    • Fix: move the --output-ptd block above the PTE write.
      • The merged program is fully built at line 720, so the order doesn't change either file's content.
      • If the large copy fails, the old pair stays intact, and the PTE becomes the last write.
    • Fix this →
  • The new # Checked above comment only points back to an earlier check (merge_ptes.py:726). That check is at lines 701-702. CLAUDE.md allows comments only for non-obvious global context.
    • Fix: replace the two one-way checks at lines 699-702 with a single check and delete the comment:
      if bool(args.input_ptd) != (args.output_ptd is not None):
          raise ValueError("--input-ptd and --output-ptd must be provided together")
    • No test asserts the current error messages.
    • Fix this →
  • The PR description doesn't match the branch.
    • It calls the move "a plain rename", but main() changes behavior. The PTD now goes through write_atomically, and the --output-ptd requires --input-ptd check runs before any write.
    • The "While moving" paragraph covers only the PTE. The Tests section mentions only the import strings, not the new mode and directory assertions.
    • It says 17 test_remap_pte_devices.py tests pass. The file defines 18, none skipped, and the CUDA backend: add a PTE device-id remap tool #25315 commit message also says 18. Is one failing on this stack, or is the count out of date?
    • The commit message names a _scripts_atomic_output writer. The module is scripts/_atomic_output.py.

Testing

  • Two behaviors this PR changes have no test (backends/cuda/tests/test_merge_ptes.py:606, :635).
    • PTD mode. test_cli_writes_merged_pte_and_reuses_base_ptd sets and checks only the PTE's mode.
      • The PTD write at merge_ptes.py:729-733 is new, and a wrong mode_from there would still pass.
      • Fix: chmod first.ptd_path to a mode different from the PTE's, such as 0o660, and assert that output_ptd has it.
    • The moved check. Nothing tests --output-ptd without --input-ptd, or any other argument check in main().
      • Fix: call merge_main() with only --input-pte, --output-pte and --output-ptd. Assert a ValueError mentioning --input-ptd, and that the output PTE was not created.
      • The code before this PR fails that test, because it wrote the PTE before raising.
    • Fix this →
    • Neither test_merge_ptes.py nor test_remap_pte_devices.py makes a write fail. So nothing in the stack tests the claim that the tool "removes its temp file when a write fails". That test belongs with _atomic_output.py in CUDA backend: add a PTE device-id remap tool #25315.

Backward Compatibility

  • The label doesn't mark the retired module path as a breaking change.
    • In .github/release.yml, release notes: devtools files this PR under "Devtools" (lines 60-63), the section for the debugger and profiler.
    • "Breaking Changes 🛠" is fed only by the breaking-change and Semver-Major labels (lines 8-11), and there is no CUDA category.
    • So readers checking the release notes for upgrade breaks won't learn that python -m executorch.backends.cuda.merge_ptes is gone.
    • Fix: add the breaking-change label. This doesn't change your decision not to keep a shim.

Recommendation

Approve: nothing blocks merging. The two tests and the label are quick to add in this PR; the write order and the comment are optional.

  • Gather context (diff vs origin/cuda-remap-pte-devices, CLAUDE.md, checklist)
  • Run independent reviews (3 reviewers)
  • Consolidate findings
  • Fact-check findings
  • Post final review

Three independent reviewers covered all five changed files, and I re-read the cited code for every finding. I did not run test_merge_ptes.py, which needs flatc, and I did not run Buck. CI covers both.

@claude

claude Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

Claude finished @Gasoonjia's task in 17m 25s —— View job


PR Review: #25323

Approve: nothing blocks, but the commit message still names a breaking-change label the PR doesn't have, two of the new CLI tests check less than they claim, and the PTD copy no longer uses the kernel's fast copy.

Detailed review

Summary

The PR moves merge_ptes into backends/cuda/scripts/, points every reference at the new module, and writes both outputs through write_atomically, PTD first. Nothing blocks, and all five findings are minor.

Except for the remap test count, everything below was raised in the 01:26 review and is still on the branch. The 01:38 follow-up only re-checked the two cleanups, so it didn't carry these forward.

Code Quality

  • The commit message and the description don't match the branch.

    • Label: the commit message says "the release-notes: devtools and breaking-change labels cover the move". The PR has no breaking-change label, and the description itself says the repo doesn't have one.
    • Test count: both say "the 20 test_remap_pte_devices.py tests still pass". At HEAD the file defines 22 tests, and the base commit's own test plan also says 22.
    • "Only content changes": the description calls the move "a rename; the only content changes are the output-writer ones below". The diff also rewraps three lines (merge_ptes.py:281-282, 289-290, 562-564) and drops a blank line before CUDA_BACKEND_ID.
    • Bullet list: the description's "Every reference now uses …" list goes against CLAUDE.md's "No bullet lists of changes".

    Fix this →

  • Writing the PTD first moves the mismatch window but doesn't close it (backends/cuda/scripts/merge_ptes.py:728-739). This one is optional.

    • Each write_atomically call runs its own os.replace (_atomic_output.py:47). The PTD is therefore already replaced when the PTE write starts.
    • If the PTE write then fails (an interrupt or CI timeout during the write or its fsync, EIO, ENOSPC), the new PTD is left next to the previous run's PTE. Before this PR, those PTE-side failures left both outputs untouched.
    • The comment at lines 725-727 is accurate for the case it names, a failed PTD copy.
    • Fix: write, fsync and chmod both temp files first, then run the two os.replace calls back to back. One way is to split write_atomically in CUDA backend: add a PTE device-id remap tool #25315 into a stage step and a commit step.

    Fix this →

Testing

  • test_cli_output_ptd_without_input_ptd_writes_nothing can't detect a write (backends/cuda/tests/test_merge_ptes.py:653-675).

    • The test passes one --input-pte and no fallback. merge_cuda_pte_files_with_provenance rejects that before writing anything ("At least two CUDA PTE inputs are required", merge_ptes.py:581-582), wherever the moved check sits. So the two assertFalse(...exists()) lines can never fail.
    • If the check moved back below the PTE write, the test would still fail, but only because assertRaisesRegex sees the merge's message instead. Nothing tests that no output is written.
    • Fix: patch executorch.backends.cuda.scripts.merge_ptes.merge_cuda_pte_files_with_provenance so the merge can't be what stops the run, and assert that it was never called. A late check would then leave model.pte behind and fail the exists() assertion.

    Fix this →

  • test_cli_failed_ptd_copy_leaves_no_output only checks the PTD-first order while the merged PTE is plain bytes (test_merge_ptes.py:703-705).

    • patch("shutil.copyfileobj", ...) replaces the function for the whole process. FileBackedData.write_to_file calls it too (exir/_serialize/_cord.py:76-78), and Cord.write_to_file uses that for any non-bytes buffer.
    • Today the merged Cord is bytes-only, because the inputs are read with read_bytes() (merge_ptes.py:189). But the CUDA backend already puts FileBackedData into named data on export (cuda_backend.py:1341, cuda_weight_collector.py:188).
    • If the merge ever passes one through, a PTE-first main() fails inside the PTE write with the same "copy failed" error. It then removes its temp file, and the test passes.
    • Fix: make the side_effect raise only when its source is the input PTD, and call the real shutil.copyfileobj otherwise.

    Fix this →

Performance

  • The PTD copy no longer uses the kernel's fast copy (backends/cuda/scripts/merge_ptes.py:733).

    • On Linux, shutil.copyfile copied with os.sendfile. shutil.copyfileobj(source, output) is a Python read/write loop.
    • The loop uses the default shutil.COPY_BUFSIZE, which is 64 KiB outside Windows (checked in this runner's Python 3.11 stdlib). The new comment itself calls this file "much larger" than the PTE.
    • The same file already reads PTDs in 8 MiB chunks (line 129), and FileBackedData copies with an 8 MiB length= (_cord.py:19, :78).
    • Fix: pass length=8 * 1024 * 1024. The slowdown is unmeasured, and on slow disks the new fsync may dominate.

    Fix this →

Recommendation

Approve: nothing blocks.

  • The commit-message fix and the two test fixes are small.
  • The buffer size is a one-argument change.
  • The two-phase write is optional.
  • Gather context (diff vs origin/cuda-remap-pte-devices, CLAUDE.md, checklist)
  • Run independent reviews (3 reviewers)
  • Consolidate findings
  • Fact-check findings
  • Post final review

Three independent reviewers covered all six changed files, and I re-read the cited code for each finding. No tests ran: I couldn't run Python or Buck in this job, and the merge suite also needs flatc. That is also why the copy-speed point is unmeasured.

@Gasoonjia
Gasoonjia force-pushed the cuda-remap-pte-devices branch from a006110 to 896b9e6 Compare November 20, 2026 01:53
@Gasoonjia
Gasoonjia force-pushed the cuda-move-merge-ptes-scripts branch from c52fa38 to 7d43a53 Compare November 20, 2026 01:53
@Gasoonjia

Copy link
Copy Markdown
Contributor Author

Restacked on the updated #25315 (atomic-writer nit fixes only; no change to the move).

@Gasoonjia

Copy link
Copy Markdown
Contributor Author

@claude /pr-review

@claude

claude Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

Claude finished @Gasoonjia's task in 19m 55s —— View job


PR Review: #25323

Approve: nothing blocks, but a failed PTE write now leaves the new PTD next to the old PTE, two of the new tests can pass without checking what they're named for, and the commit message and description still make a few claims that don't match the change.

Detailed review

Summary

The PR moves merge_ptes into backends/cuda/scripts/, points every reference at the new module, writes both outputs through write_atomically with the PTD first, and registers test_merge_ptes with Buck. All six findings below are minor.

Code Quality

  • A failed PTE write now leaves the new PTD next to the old PTE (backends/cuda/scripts/merge_ptes.py:725-739). This one is optional.
    • write_atomically replaces each file as soon as that file is written (_atomic_output.py:47), and the PTD now goes first.
    • When re-running into an existing output directory, any PTE-side failure leaves the new weights next to the previous run's PTE. Examples: Ctrl-C or a CI timeout during the PTE write or its fsync, ENOSPC, or an --output-pte directory that can't be created.
    • Before this PR the PTE went first, so these failures left the old pair intact.
    • The comment at lines 725-727 is accurate for the case it names, a failed PTD copy. This is the reverse case. The merged PTE holds every variant's .so, so its write isn't negligible, and no test covers it.
    • Fix: stage both temp files (write, fsync, chmod), then os.replace them back to back. That needs a two-phase form of write_atomically in CUDA backend: add a PTE device-id remap tool #25315. If that's more than you want for this stack, leave it. Fix this →
  • The commit message and description still don't match the change.
    • Labels. The commit message says "the release-notes: devtools and breaking-change labels cover the move". The PR has no breaking-change label, and the description says the repo has none.
    • Test count. Both say 20 test_remap_pte_devices.py tests pass. The file has 22 tests and no skips, and the CUDA backend: add a PTE device-id remap tool #25315 commit message also says 22. Were they re-run after the restacks?
    • "Only the output-writer changes." The description says the only content changes are the output-writer ones. The diff also rewraps lines 281-282, 289-290 and 562-564, and drops the blank line before CUDA_BACKEND_ID (line 46).
      • The wraps aren't needed for flake8: .flake8 ignores both E501 and B950 (lines 6 and 42).
      • Revert them or mention them.
    • Symlinks. Neither text says that an existing symlink at --output-ptd is now replaced by a regular file (os.replace, _atomic_output.py:47) instead of being written through, as shutil.copyfile did. Anyone who links the output PTD to shared storage will see the shared copy stop updating.
    • Bullet list. The "Every reference now uses …" bullet list goes against CLAUDE.md's "No bullet lists of changes".
  • Nit: self.assertIsNotNone(first.ptd_path) (backends/cuda/tests/test_merge_ptes.py:605) duplicates the assert on line 606, which is the one mypy needs.

Testing

  • test_cli_failed_ptd_copy_leaves_no_output starts from an empty output directory (backends/cuda/tests/test_merge_ptes.py:677-708).
    • So it never checks the description's claim that a failed copy leaves the existing outputs untouched. A writer that deleted or truncated an existing aoti_cuda_blob.ptd before failing would still pass.
    • The sibling test_failed_atomic_write_keeps_existing_output (test_remap_pte_devices.py:372-390) seeds a previous output for exactly this.
    • The patch replaces shutil.copyfileobj for the whole process, and Cord.write_to_file uses that function for file-backed buffers (exir/_serialize/_cord.py:76-78). The test pins the PTD-first order only while the merged PTE holds plain bytes, which is true today.
    • Fix: write sentinel model.pte and aoti_cuda_blob.ptd files into output/ first, and make the patched copy raise only for the input PTD. Afterwards, assert that both sentinels are byte-identical and are the only entries. Fix this →
  • The two assertFalse lines in test_cli_output_ptd_without_input_ptd_writes_nothing can never fail (test_merge_ptes.py:674-675).
    • The test passes a single --input-pte with no fallback. Whatever the check order, merge_cuda_pte_files_with_provenance raises "At least two CUDA PTE inputs are required" (merge_ptes.py:581-582) before anything is written.
    • The test does fail if the check moves back below the writes, but only because the error message changes. The failure would then point at the input count, not at the write order.
    • Fix: patch merge_ptes.merge_cuda_pte_files_with_provenance and assert that it was not called. Fix this →

Performance

  • The PTD copy lost the kernel fast path (merge_ptes.py:730-735).
    • On Linux, shutil.copyfile copied with os.sendfile.

    • shutil.copyfileobj(source, output) is a Python read/write loop with the default buffer, which is 64 KiB on Linux through Python 3.13. It runs over what the comment calls the much larger file.

    • I couldn't run Python here, so I haven't measured the slowdown.

    • Fix: pass FileBackedData(str(args.input_ptd[0])).write_to_file as the writer, importing it from exir/_serialize/_cord.py as backends/cuda/cuda_weight_collector.py:20 already does.

      • It copies in 8 MiB chunks (_cord.py:19, :76-78).
      • It also removes the with block, the lambda and the shutil import.
      • It ends the reuse of source, which is an open file at line 730 and a string at lines 747 and 750.

      Fix this →

Recommendation

Approve: nothing blocks. The two test fixes, the copy change and the text edits are small enough to land in this PR. The two-phase write is optional.


  • Gather context (diff vs origin/cuda-remap-pte-devices, CLAUDE.md, checklist)
  • Run independent reviews (3 reviewers)
  • Consolidate findings
  • Fact-check findings
  • Post final review

Three independent reviewers covered all six changed files, and I re-read the cited code for every finding. I left out three reviewer findings:

  • the CLI's ValueError style differs from remap_pte_devices, but that predates this PR and matches the rest of the file;
  • adding test_merge_ptes.py to pytest.ini, which I left out because the test imports triton through cuda_weight_collector;
  • a type mismatch in the write_atomically signature, which belongs to CUDA backend: add a PTE device-id remap tool #25315.

Python, flatc and Buck aren't available in this job, so no tests ran and the copy-speed point is unmeasured.

@Gasoonjia

Copy link
Copy Markdown
Contributor Author

@claude addressed the small items from this round as well, except the optional two-phase write:

  • The PTD copy now goes through FileBackedData (8 MiB chunks, so no slowdown versus shutil.copyfile); the shutil import is gone.
  • The failed-copy test now seeds sentinel model.pte/aoti_cuda_blob.ptd outputs and asserts both stay byte-identical, and the missing---input-ptd test now also asserts the merge itself was never called (and needs no real artifact, so it passes without flatc).
  • Description and commit message now match the change, including the PTD/symlink/mode behavior, the helpers' names, the 22-test count, and the three wrapped overlong lines. The duplicate assertIsNotNone and the back-pointer comment are dropped.

Two-phase staging of both outputs stays out, as you flagged it optional: it needs a two-phase form of the base PR's writer, and PTD-first already ensures the likelier failure (the large copy) cannot orphan a new PTE.

@Gasoonjia

Copy link
Copy Markdown
Contributor Author

@claude /pr-review

@claude

claude Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

Claude finished @Gasoonjia's task in 15m 38s —— View job


PR Review: #25323

Approve: nothing blocks, but the advertised --output-ptd symlink change has no test, and the description says the new PTD copy is as fast as the old shutil.copyfile, which the code doesn't support.

Detailed review

Summary

The PR moves merge_ptes into backends/cuda/scripts/, points every reference at the new module, and writes both outputs through write_atomically, PTD first via FileBackedData. Nothing blocks, and all three findings below are minor.

Code Quality

  • The description's "as fast as the old shutil.copyfile path" claim isn't supported.
    • The PR gives no measurement, and two things in the code point the other way:
      • On Linux, shutil.copyfile copied in the kernel with os.sendfile. FileBackedData.write_to_file copies through user space with shutil.copyfileobj (exir/_serialize/_cord.py:76-78).
      • The old copy returned once the data was in the page cache. write_atomically now fsyncs the whole PTD before the rename (backends/cuda/scripts/_atomic_output.py:44-45), so the tool waits for the multi-GB weights to reach disk.
    • The description also leaves out a space cost. When the output PTD already exists, the temp file needs room for a second full copy until the rename. The old copy truncated the existing file first.
    • Fix: drop "as fast as the old shutil.copyfile path", or replace it with the trade-off: a durable write, the fsync wait, and room for a second copy. Fix this →
  • The PR may be missing CLAUDE.md's AI disclosure. CLAUDE.md asks AI-authored PRs to say so and name the tool. Neither the description nor the commit message does. Low confidence: nothing in the PR shows whether an AI tool helped write it, so ignore this if none did.

Testing

  • The advertised --output-ptd symlink change has no test (backends/cuda/scripts/merge_ptes.py:728-733, backends/cuda/tests/test_merge_ptes.py).
    • The description says an existing --output-ptd symlink "is now replaced by a regular file rather than written through". Users will notice: shutil.copyfile used to overwrite the link's target, and os.replace (_atomic_output.py:47) now swaps the link for a regular file.

    • No test in backends/cuda/tests/ creates a symlink, including the base PR's writer tests in test_remap_pte_devices.py.

      • So a change in either direction passes the suite. For example, passing args.output_ptd.resolve() to write_atomically "to keep the user's link" would quietly bring back writing through the link.
      • The resolve() skip at line 728 is untested too. When --output-ptd is --input-ptd[0] or links to it, nothing is written and the link stays.
    • Fix:

      • add a CLI case where output_ptd is a symlink to a sentinel file outside output/, and assert that the sentinel is byte-identical afterwards and that output_ptd is a regular file holding the first input PTD's bytes;
      • optionally, add a case that links output_ptd to first.ptd_path and asserts the link is kept.

      Fix this →

Recommendation

Approve: nothing blocks. The test and the description edit are small.


  • Gather context (diff vs origin/cuda-remap-pte-devices, CLAUDE.md, checklist)
  • Run independent reviews (3 reviewers)
  • Consolidate findings
  • Fact-check findings
  • Post final review

Three independent reviewers covered all six changed files, and I re-read the cited code for each finding. I dropped two reviewer suggestions:

  • Adding test_merge_ptes.py to pytest.ini. The suite imports triton (via cuda_weight_collector → passes/lower_offgraph_kv.py → triton/kernels/sdpa.py), and a CPU-only job may not have it.
  • A weakness in the 0o640 mode check that only shows up on hosts with umask 027.

No tests ran: I couldn't run Python, flatc or Buck in this job, so the copy-speed point is unmeasured.

@claude

claude Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

Claude finished @Gasoonjia's task in 44s —— View job


Follow-up review: #25323

Approve: all three items are on the branch and work as you describe, and nothing blocks merging; one optional nit is left in the details.

Details

What I checked

  • PTD copy via FileBackedData (backends/cuda/scripts/merge_ptes.py:728-733)
    • FileBackedData.write_to_file copies with shutil.copyfileobj(..., length=8 MiB) (exir/_serialize/_cord.py:76-78), so the 64 KiB default-buffer concern is gone. import shutil is removed.
    • It's safe to wrap the input this way. The constructor defaults to cleanup=False (_cord.py:20-30), so no finalizer is registered and the input PTD is never deleted. The constructor's os.path.getsize runs before write_atomically, so a missing input fails before any temp file exists.
    • The variable that was reused as both an open file and a str is gone too.
  • Failed-copy test (test_merge_ptes.py:686-731)
    • It now seeds previous pte / previous ptd and asserts both are byte-identical and are the only entries afterwards. That covers the temp-file cleanup and the "existing outputs untouched" claim.
    • The patch now targets merge_ptes.FileBackedData, not process-wide shutil.copyfileobj. So the PTE's Cord.write_to_file is unaffected even if the merged Cord ever holds file-backed chunks. A PTE-first main() would overwrite model.pte and fail the test, so the ordering is now pinned for the right reason.
  • Missing --input-ptd test (test_merge_ptes.py:652-681): merge.assert_not_called() plus the two assertFalse(exists()) lines now really test that the check runs before the merge and before any write. As you said, it needs no real artifact or flatc.
  • Commit message: it now says only release notes: devtools covers the move, gives 22 remap tests, names :scripts_atomic_output, and mentions the PTD-first order, the modes and the three wraps. The duplicate assert and the back-pointer comment are gone. No in-repo references to the old backends.cuda.merge_ptes path remain.

Optional nit

  • test_cli_failed_ptd_copy_leaves_no_output now checks that the previous outputs stay. A name like ..._keeps_existing_output (to match test_failed_atomic_write_keeps_existing_output in the remap suite) would describe it better.

Two-phase write

I accept leaving it out. PTD-first covers the likelier failure, the large copy.

Verification

This was a targeted re-check of the items you addressed, not a full three-reviewer pass. I read the diff against origin/cuda-remap-pte-devices, plus main() and FileBackedData. I didn't run the tests (the suite needs flatc) or Buck; CI covers both.

  • Confirm the fixes are on the branch
  • Check the FileBackedData PTD copy
  • Check the two updated tests
  • Check the commit message
  • Post follow-up

Stacked on the remap_pte_devices PR, which starts the scripts
directory. merge_ptes is the other standalone CUDA backend tool, so
it belongs there too: backends/cuda/merge_ptes.py moves to
backends/cuda/scripts/merge_ptes.py. The CUDA CI workflow, the CUDA
examples README, its tests, and the :merge_ptes Buck target now use
executorch.backends.cuda.scripts.merge_ptes. The old module path is
retired outright; the release-notes: devtools label covers the
move.

While moving, both outputs write through the shared atomic writer
from the base PR's :scripts_atomic_output library. The PTD is
copied first with FileBackedData, so a failed copy leaves the
existing output PTE untouched. Both outputs keep their inputs'
permissions and remove their temp files when a write fails, instead
of leaving 0600 or truncated files behind. The --output-ptd
argument check now runs before any output is written. Three
overlong lines in the moved file are wrapped as part of the move.

Test Plan:
- backends/cuda/tests/test_merge_ptes.py (now also registered as a
  Buck python_unittest): import strings point at the new module
  path, the CLI test checks PTE and PTD output permissions against
  distinctive input modes, --output-ptd without --input-ptd is
  rejected before the merge runs, and a failed PTD copy leaves prior
  outputs unchanged with no temp files. The suite needs flatc,
  which this environment lacks, and runs in CI; the missing-input
  argument test passes locally.
- backends/cuda/tests/test_remap_pte_devices.py still passes (22
  tests) on this stack.
@Gasoonjia

Copy link
Copy Markdown
Contributor Author

Restacked onto current main after #25315 landed in 21c7c9946e. The branch is now the single merge_ptes move commit on top of that merge commit; local verification: the 22 remap tests plus the new flatc-free merge argument test pass, flake8 and Black are clean on the moved files. The rest of the merge suite still needs CI's flatc.

@Gasoonjia

Copy link
Copy Markdown
Contributor Author

@claude /pr-review

@claude

claude Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

Claude finished @Gasoonjia's task in 13m 22s —— View job


PR Review: #25323

Approve: nothing blocks, but one new test's name says the opposite of what it checks, the PR description and commit message are out of date in a few places, and the --output-ptd symlink change that the description advertises has no test.

Detailed review

Summary

The PR moves merge_ptes into backends/cuda/scripts/ and points every reference at the new module. It writes both outputs through write_atomically, PTD first, and registers test_merge_ptes with Buck. Nothing blocks; the three findings below are minor.

Code Quality

  • test_cli_failed_ptd_copy_leaves_no_output says the opposite of what it checks (backends/cuda/tests/test_merge_ptes.py:683).

    • The test first writes previous pte and previous ptd into output/ (lines 696-699).
    • It then asserts that both files are still there, byte-identical, and the only entries (lines 726-731). It never checks a case where no output is left.
    • Anyone triaging a failure under this name will look for the wrong behavior.
    • Fix: rename it to match the sibling test_failed_atomic_write_keeps_existing_output (test_remap_pte_devices.py:372), e.g. test_cli_failed_ptd_copy_keeps_existing_outputs.

    Fix this →

  • The PR description and commit message are out of date in a few places. The commit message is the text that lands in git log.

    • Stacking. CUDA backend: add a PTE device-id remap tool #25315 has landed as 21c7c99.
    • Copy speed. The description says the PTD copy is "as fast as the old shutil.copyfile path". That isn't supported:
      • On Linux, shutil.copyfile copies in the kernel with sendfile and never fsyncs.
      • FileBackedData.write_to_file copies through user space (exir/_serialize/_cord.py:76-78).
      • write_atomically then fsyncs the whole PTD (backends/cuda/scripts/_atomic_output.py:44-45), so the tool now waits for the multi-GB weights to reach disk.
    • Disk use. Neither text says that the temp file sits next to the existing output until os.replace (_atomic_output.py:37-47).
      • Re-merging into an existing output directory therefore needs free space for a second full PTD. The old shutil.copyfile truncated the existing file first.
      • On a disk with room for only one copy, a re-run that used to succeed now fails with ENOSPC. The old outputs stay intact.
      • Replace the speed claim with these two trade-offs.
    • Label. The commit message says "the release-notes: devtools label covers the move".
      • The label is actually release notes: devtools, and git log readers can't see labels anyway.
      • Say what users have to change instead: python -m executorch.backends.cuda.merge_ptes is now python -m executorch.backends.cuda.scripts.merge_ptes.
    • Test command. CONTRIBUTING.md:403-406 asks for repeatable test instructions, for example the command you ran. Neither text gives one.

    Fix this →

Testing

  • The --output-ptd symlink change that the description advertises has no test (backends/cuda/scripts/merge_ptes.py:727-733).

    • The description says an existing --output-ptd symlink "is now replaced by a regular file rather than written through".
      • shutil.copyfile followed the link and overwrote its target.
      • os.replace (_atomic_output.py:47) replaces the link itself with a regular file.
      • Users who link the output PTD to shared storage will find the shared copy no longer updates.
    • Neither test_merge_ptes.py nor test_remap_pte_devices.py creates a symlink, so the suite passes whichever way this behaves.
    • Fix: add a CLI case where output_ptd is a symlink to a sentinel file outside output/. Assert that the sentinel is unchanged, that output_ptd is no longer a symlink, and that it holds the first input PTD's bytes.

    Fix this →

Recommendation

Approve: nothing blocks. The rename and the text edits are quick, and the symlink test is small.


  • Gather context (diff vs origin/main after the restack, CLAUDE.md, checklist)
  • Run independent reviews (3 reviewers)
  • Consolidate findings
  • Fact-check findings
  • Post final review

Three independent reviewers covered all six changed files, and I re-read the cited code for every finding. I left out three reviewer findings:

  • merge_ptes reports argument errors as a ValueError traceback, while remap_pte_devices uses parser.error (raised by two reviewers). This predates the PR and matches the rest of the file.
  • The PTE's mode_from uses sources[0].pte_path rather than args.input_pte[0]. Both are the same path.
  • The blank line dropped before CUDA_BACKEND_ID is formatting.

No tests were run: the merge suite needs flatc, and I didn't run Buck. So the copy-speed point is unmeasured.

This branch is being deployed

1 in progress deployment
cadence — f7cff090 Deployed Nov 20, 2026 by Gasoonjia via hifi-op-test / hifi4 #39584
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. release notes: devtools Changes to dev tooling, for example the debugger & profiler

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant