Skip to content

Models: a fallback model is for that run only, never the saved choice - #2614

Merged
AbirAbbas merged 4 commits into
devfrom
fix/fallback-model-run-only
Nov 13, 2026
Merged

AbirAbbas merged 4 commits into
devfrom
fix/fallback-model-run-only

Conversation

@AbirAbbas

@AbirAbbas AbirAbbas commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Carries #2465, which was reviewed and merged into workstreams/main but never reached dev. Its integration PR, #2306, is closed: its base app/v3 was retired by #2561, and #2597 left it out. The commit applied to dev without conflict and nothing was dropped; its size is the fix itself (about 570 lines of code across session, the linked-local wire, the terminal and the app, and about 1,900 lines of tests).

The fault. On a local terminal launch whose configured model is not in the listed catalog (for example no key for its provider while a local Ollama is running), the terminal picks the first listed model so the conversation can run. It switched through the ordinary model setter, so the fallback became the conversation's saved model. The agent then reopened on it, in the desktop app too.

The change: a fallback is for that run only.

  • session.Agent.UseModelForNow / ConfiguredModel (new, additive). Requests use the fallback; the saved model, provider, identity and any main/fallback pair keep naming the configured choice. An explicit SetModel / SetConversationModels clears the run-only model and saves as before.
  • The linked-local wire carries the new door (internal/remote). An older engine without it keeps the old setter behaviour.
  • Terminal: the availability reconcile and the move after disconnecting the running model's service use the run-only door, off the update loop. Every call that starts a turn waits for that agent's pending switch, so the engine sees the switch before the message. Later reconciles return to the configured model as soon as it is listed.
  • A service rename relabels the saved choice and a run-only model separately. start_chat without a model gives the new chat its parent's configured model.
  • The app reports the configured model on agents.list / agents.open.
  • Unchanged: a fresh service connect still saves; the credit-default follow; provider fallback-chain rescue; remote --host doors; the continued-chat replacement for a chat opened on another computer.

The original PR's validation contract (C1 to C6) and its test names are in #2465.

Evidence that dev still has the fault

dev (80b5861) has no run-only door (UseModelForNow, ConfiguredModel do not exist). A throwaway probe on clean dev, with a fake agent that records calls to the saving setter and a catalog that lacks the configured model:

--- FAIL: TestProbeAvailabilityFallbackUsesTheSavingSetter (0.00s)
    zz_probe_fallback_test.go:31: the automatic replacement called the conversation's saving SetModel with ["vendor/replacement"]

The probe is not part of this PR; TestListedModelReplacementKeepsProfileAndReturnsToConfiguredModel and TestRunOnlyModelFailureDoesNotCallThePersistentSetter cover the same ground here.

I also checked that dev added no turn-starting call in internal/tui3 since the original was written: all five Submit / SubmitImage roads sit behind waitModelSelection.

Tests

CI on 0d9715b: every check passes on the first run (touched codeaf, touched session, touched tui3, touched rest, light gate, Windows native tests). TestProcessCloseJoinsTheUsageWriter and TestDoReturnsAfterCancelledAccountingAndItsWriterStop were classified flaky, as on many unrelated runs in #1749.

Local, macOS, focused runs with the fix: pass.

  • internal/tui3: the 85 tests in the new and touched model test files plus TestNoEngineDoorIsAskedFromTheUpdateLoop: ok.
  • internal/session: the new tests plus -run 'ConversationModel|ModelIdentity|SetModel|Fallback': ok.
  • ./internal/remote/: ok (whole package).
  • cmd/codeaf: TestAppReopensConfiguredModelAfterRunOnlyReplacement, TestLinkedLocalSourceRefreshKeepsRunOnlyModelTemporary: pass.

Local make pr-ready did not pass on this Mac. The PR is marked ready on the strength of CI and the comparison with dev below. The light gate and touched rest passed. The heavy legs are red here for reasons that are red or flaky on unmodified dev (80b5861) on the same machine:

  • touched session: 7 failures. 5 fail on dev here too (TestACommandLookFailsOnAFailingPipeStage, TestWatchTickFailsOnAFailingPipeStage, TestBashBeltTruncationCapFollowsTheWindow, TestBashBeltTruncationWritesHeadTailAndPath, TestTheFixedPrefixStaysUnderItsBudget). TestAdmittedLegacyProjectCacheKeepsItsIdentityAndBytes and TestExplicitInPlaceWorkerRoutesScratchWithoutOwningTheProject pass alone on this branch; the first also failed once on dev.
  • touched tui3: 1 failure, TestContinueCardSendsAndSavesWhileStorageOff (turns=2 writes=0). Run 40 times alone it fails 1 of 40 on this branch and 4 of 40 on dev, with the same message.
  • touched codeaf: 12 failures. 9 are Factory and memory-worker tests that fail on dev here too, alone. The other 3 pass when run alone on this branch.

Changes after review

Two commits on top of 0d9715b: da950e4 and 00030a0. The CI result in Tests above is for 0d9715b; the new head is 00030a0.

  1. A run-only replacement now stays put while it is listed.

    • Found: while the configured model was unlisted, every availability reconcile worked the replacement out again as the catalog's first row. So right after a service disconnect, the chat default the terminal had just named was quietly swapped for catalog row 0, and any later catalog refresh that put a different row first moved the conversation again between turns. On dev the running model was kept.
    • Changed: ensureAvailableModel (internal/tui3/modeldefault.go) now follows one rule, runOnlyModel: the configured model if it is listed, else the model already running (or pending) while it is listed, else the first listed row. The manual (models-and-cost.md, services.md) and the change entry say so.
    • Tests: TestDisconnectReplacementIsTheModelTheNextRequestUses uses a real session.Agent and checks the disconnect receipt, the surface, engine and saved models, and the model in the next real provider request. TestRunOnlyReplacementStaysPutWhenTheCatalogReorders checks that a reorder does not move the run, and that once the replacement is delisted the run falls back to the first row. Both fail on 0d9715b and pass now.
  2. start_chat runs the new chat on its parent's replacement and names that model.

    • Found: a chat started without a model saves the parent's configured model. When that model was unlisted, the new chat took the catalog's first row instead of the replacement the parent was running, and the started chat · <model> receipt named the parent's unavailable configured model, not the one the new chat used.
    • Changed: the new chat picks with the same runOnlyModel rule, using the parent's running model (internal/tui3/startchat.go, new runningModel helper). It still saves the parent's configured model, and the receipt names the model it runs on. conversations-and-teams.md and the change entry say so.
    • Tests: TestRequestedChatRunsOnAndNamesItsParentsReplacement fails on 0d9715b and passes now. TestRequestedChatSavesConfiguredModelBeforeItsOwnAvailabilityReconcile still passes.
  3. start_chat is asked beside the door line again, as on dev.

    • Found: this PR had moved the start from besideLine to the ordered door line. Opening the new conversation (dialing an engine, seating its model and sending its first message) then held up anything a person pressed meanwhile, such as answering a question, picking a model or switching chats.
    • Changed: back to a.besideLine, with a comment giving the reason: like a team start, it is not a person's gesture and nothing in it needs the parent's order.
    • Tests: TestRequestedChatDoesNotHoldTheOrderedDoorLine blocks the start and checks that a door queued meanwhile runs within 2 s. It fails on 0d9715b (after 2.01 s) and passes now and on dev. TestOnlyReadsNobodyPressedForAreAskedBesideTheLine still passes.

No review item was left unchanged. PROTOCOL.md and internal/session/prompts/system.md did not need edits: no wire or tool semantics changed.

🤖 Generated with Claude Code

AbirAbbas and others added 2 commits November 12, 2026 12:09
Carries #2465 from the closed integration PR #2306 (workstreams/main);
it never reached dev. Applied without conflict.

Co-Authored-By: Claude Mythos 5.2 <[email protected]>
AbirAbbas and others added 2 commits November 12, 2026 18:25
ensureAvailableModel re-derived the replacement from the configured model
on every reconcile. While that model was unlisted it took the first listed
row, so the replacement a disconnect picked and named (the chat default)
was overridden at once by catalog row 0, and any refresh that listed a
different row first moved the conversation quietly between turns.

The reconcile now keeps the configured model whenever it is listed, then
the model already running (or pending) while it is listed, and only then
falls back to the first listed row. The manual and the change entry say
so.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…beside the door line

A chat started with no model saves the parent's configured model. While
that model was unlisted, the child took the catalog's first row rather
than the replacement the parent was running, and the started chat
receipt named the parent's unavailable configured model. The child now
uses the same rule as the parent's reconcile (configured if listed, else
the parent's running replacement if listed, else the first row), and the
receipt names the model the child runs on.

The re-cut had also moved the start from besideLine to the ordered door
line, so the new conversation's dial, model seating and first message
held up any answer, model pick or chat-switch replay a person pressed
meanwhile. Like a team start it is not a person's gesture and nothing in
it needs the parent's order, so it is asked beside the line again, as on
dev.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@AbirAbbas
AbirAbbas merged commit bd27c62 into dev Nov 13, 2026
10 checks passed
@AbirAbbas
AbirAbbas deleted the fix/fallback-model-run-only branch November 13, 2026 00:38
AbirAbbas added a commit that referenced this pull request Nov 13, 2026
Brings in #2613, #2612, #2614, #2618 and #2615. #2618 changes the
visibility check this branch's job-events cursor reads: a spent GitHub
rate limit returns GitHub's retry time instead of a cached denial, and is
remembered per token until then.

The two meet in repoAllowed, which this branch made answer (allow, known).
A remembered limit answers false, false: a limit says nothing about the
repository, so the job-events page ends before it as before any undecided
one, as a fresh limit already does (askRepo's cache=false).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant