Repository navigation
fix: idle Factory polls read only job documents that changed - #2361
Closed
santoshkumarradha wants to merge 13 commits into
Closed
santoshkumarradha wants to merge 13 commits into
santoshkumarradha wants to merge 13 commits into
Conversation
Co-Authored-By: Claude Opus 5.5 <[email protected]>
List stats every numeric job file on every call and copies its cached JSON graph before migration, retention or caller mutation. Locked reads and writes bypass the cache. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Keep file-identity reuse and detached List results. Prove external changes, retention, migration and concurrent looks. Derive thread-only authority under the item lock so a Stop during selection cannot restart job steps. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
An Update callback holding an item lock could deadlock against a List migrating that item. Published indexes are never changed, so a scan now reads a snapshot and the lock guards only the pointer. Co-Authored-By: Claude Opus 5.5 <[email protected]>
File identity cannot see an in-place rewrite that keeps its size and time, and a Stop or budget must not rest on file times with no limit. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]> # Conflicts: # internal/factory/lease/lease.go
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Collaborator
|
Closing because this is already on
The branch stays as it is, so nothing is lost. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Idle Factory polls no longer read every job document on each look. Held: do not merge until the owner has reviewed before-and-after numbers and authorized it. Stacked on #2358; retarget after that lands.
What changed
With #2358 applied, the largest remaining engine work on a real install with 355 jobs is
store.List. A read-only sample of the idle engine shows 135 samples inList: leasePull82, job settlement 35, runner capacity 12,OverLimit11. Each look decoded every document.Listnow stats every file on every look and decodes only files whose identity, size or time changed. It returns a deep copy, so callers, migration and output retention never touch the kept document.Listreads every document again at least once a minute, so a decision never rests on file times for longer than that.Measurements
Read counts on a floor of 300 jobs, unchanged floor inside the 60 s bound, from the tests:
PullOverLimitReal app, Mac Studio (M3 Ultra), 355 jobs, 3 job chats open, about 4 minutes after start. Two runs each, same desktop build, same session, including the 60 s re-read. Not the laptop. Before is
56d8b5acb(#2358 without its re-read); after is4aa44a98f(this PR and #2358's re-read). The later merges of app/v3 were not re-measured.The gain on this machine is small: about 10% less engine CPU for about 14-17 MB more memory. Typing does not change. The target is the laptop, where this work is a larger share.
Laptop before and after: not measured yet. It needs an owner-approved install. This PR stays draft until those numbers are here.
Correctness tests
TestPullAndCapacitySeeExternalFloorChanges,TestSettleSeesExternalStopStartAndBudgetTestPullRechecksLockedJobAfterListing,TestSettleRechecksStopAndBudgetAfterListing; the lease fix itself landed separately in #2362TestListOwnsMutableDocuments,TestListReappliesOutputRetention,TestListReappliesRepositoryMigrationTestListSeesRenameWithSameSizeAndTime-raceTestFloorLooksRaceExternalWritesLimits
Listreads every document again at least once a minute, the bound, tested byTestListSeesInPlaceSameTimeEditWithinItsBound. Grants still read under the lock.🤖 Generated with Claude Code