Skip to content

diff: use ses for Myers sequence comparison - #418

Open
shadowmkj wants to merge 9 commits into
uutils:mainfrom
shadowmkj:feat/myers-ses-implementation
Open

shadowmkj wants to merge 9 commits into
uutils:mainfrom
shadowmkj:feat/myers-ses-implementation

Conversation

@shadowmkj

@shadowmkj shadowmkj commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

PR to evaluate ses (Shortest Edit Script) for sequence comparison using Eugene Myers' diff algorithm, and to compare CodSpeed
performance and hunk compatibility against main and #415 (similar).

ses is the minimal crate implementation of Myers' diff algorithm. https://github.com/shadowmkj/ses

Unlike similar, ses produces hunks that are 1:1 compatible with GNU diff's alignments and tie-breaking .

What changed:

  • Replaced the diff crate dependency with ses across normal_diff, unified_diff, context_diff,
    ed_diff, and side_diff.
  • Added edge-case tests in tests/by-util/test_diff.rs verifying 1:1 GNU diff compatibility
    (symmetrical/asymmetrical ties, replacements, and boundary changes).

- Replace the external `diff` crate dependency with the `ses` crate across all diff modules
- Update pattern matching in context, ed, normal, side, and unified diff implementations to use `ses::Change` variants (`Delete`, `Insert`, `Equal`)
@shadowmkj
shadowmkj marked this pull request as ready for review November 19, 2026 14:52
@codecov

codecov Bot commented Nov 19, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 0.00%. Comparing base (b93330b) to head (c626797).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@     Coverage Diff     @@
##   main   #418   +/-   ##
===========================
===========================

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

tridev-tej and others added 3 commits November 20, 2026 04:34
empty lines are written without the blanks of their marker in the normal, context and unified formats. the context colors still apply to them.
@sylvestre

Copy link
Copy Markdown
Collaborator

sorry but it is unlikely that we will take add this dependency over similar

@shadowmkj

Copy link
Copy Markdown
Contributor Author

That's what I want to know, since similar cannot produce GNU compatible hunks? What is the solution that you are suggesting?

This crate is just temporary and for isolating the core algorithm from this repository (so I can research and play around), and it's small enough to be merged into diffutils itself. That's what I mentioned here: #415 (comment)

@shadowmkj
shadowmkj force-pushed the feat/myers-ses-implementation branch from c626797 to 143b9df Compare November 20, 2026 10:50
@codspeed

codspeed Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will degrade performance by 10.85%

3 warnings, 1 note

⚠️ 32 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚠️ 16 benchmarks with a significant change ran on different hardware

Their base and head results were measured on machines with a different CPU, core count or memory, so the changes might not come from the code. Run base and head on the same runner type.

Learn more about Macro Runners

See affected benchmarks

⚠️ 16 benchmarks with a significant change ran in a different software environment

Their base and head results were measured with a different OS or toolchain, such as compiler, runtime or library versions, so the changes might come from this difference rather than from the code. Open an affected benchmark to see what changed.

See affected benchmarks

ℹ️ This run is excluded from the branch history

This commit was force-pushed, so this run no longer appears in the branch's performance history.

❌ 16 regressed benchmarks
✅ 10 untouched benchmarks
🆕 16 new benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
❌ cmd_cmp_gnu_equal[1000] 22.8 µs 26.4 µs -13.74%
❌ cmd_cmp_gnu_equal[100] 22.9 µs 26.3 µs -12.92%
❌ cmd_cmp_gnu_equal[25000] 23.1 µs 26.3 µs -12.47%
❌ cmd_cmp_gnu_equal[10000] 23.1 µs 26.3 µs -12.47%
❌ cmd_cmp_release_equal[25000] 24.2 µs 27.4 µs -11.56%
❌ cmd_cmp_release_equal[1000] 24.3 µs 27.3 µs -10.97%
❌ cmd_diff_release_equal[25000] 24.7 µs 27.6 µs -10.83%
❌ cmd_diff_release_equal[10000] 24.7 µs 27.7 µs -10.81%
❌ cmd_cmp_release_equal[100] 24.3 µs 27.2 µs -10.66%
❌ cmd_cmp_release_equal[10000] 24.4 µs 27.2 µs -10.41%
❌ cmd_diff_gnu_equal[100] 23.6 µs 26.2 µs -10.16%
❌ cmd_diff_gnu_equal[1000] 23.6 µs 26.3 µs -10.16%
❌ cmd_diff_gnu_equal[10000] 24.1 µs 26.8 µs -10.07%
❌ cmd_diff_release_equal[1000] 24.8 µs 27.5 µs -9.84%
❌ cmd_diff_gnu_equal[25000] 24.2 µs 26.7 µs -9.51%
❌ cmd_diff_release_equal[100] 25.3 µs 27.2 µs -6.86%
🆕 cmd_diff_gnu_different[100] N/A 26.3 µs N/A
🆕 cmd_diff_gnu_different[1000] N/A 26.6 µs N/A
🆕 cmd_diff_gnu_different[10000] N/A 27.1 µs N/A
🆕 cmd_diff_gnu_different[25000] N/A 27 µs N/A
... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing shadowmkj:feat/myers-ses-implementation (8745325) with main (824dfcf)

Open in CodSpeed

@shadowmkj
shadowmkj force-pushed the feat/myers-ses-implementation branch 2 times, most recently from d661fee to 94f9bb6 Compare November 20, 2026 12:13
binhvq and others added 3 commits November 20, 2026 18:55
…ls (uutils#384)

* diff: count -t columns like a terminal after CR, backspace and controls

* tests: port the -t control character test to uutests

* utils: keep the do_expand_tabs doc comment in one piece

* utils: let do_write_line write the line marker
- Check for identical content early in compare_files to bypass binary checks
- Return empty output immediately in normal and unified diff generators when inputs are identical

perf(utils): optimize file equality checks with sampling heuristic

- Add `is_identical` helper to `utils` for O(1) early rejection of non-identical files
- Sample tail, midpoint, and quarter points for large buffers ($\ge$ 256 bytes) before falling back to full comparison
- Update diff modules (`diff`, `normal_diff`, `unified_diff`) to utilize the new optimized equality check
- Add comprehensive unit tests covering empty, small, and large-buffer sampling rejections
@shadowmkj
shadowmkj force-pushed the feat/myers-ses-implementation branch from 94f9bb6 to e3a3524 Compare November 20, 2026 13:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants