Skip to content

fix(engine): stop replaying an abandoned partial turn on retry - #363

Open
ChakrawarShubham wants to merge 1 commit into
andrewyng:mainfrom
ChakrawarShubham:fixGeminiTrailingModelTurn
Open

fix(engine): stop replaying an abandoned partial turn on retry#363
ChakrawarShubham wants to merge 1 commit into
andrewyng:mainfrom
ChakrawarShubham:fixGeminiTrailingModelTurn

Conversation

@ChakrawarShubham

@ChakrawarShubham ChakrawarShubham commented Jul 31, 2026

Copy link
Copy Markdown

Fixes #369

Updated: this PR originally patched the Gemini converter. After opening it I tested Anthropic, found the same wedge there and worse, and moved the fix to the engine where the defect actually is.

Problem

A session whose turn dies mid-stream becomes permanently unrecoverable from the UI — every Retry returns the same 400. Hit on Gemini 3.6 Flash: the agent streamed some text, the turn died to a 503 UNAVAILABLE, and Retry 400s forever after.

Error: 400 INVALID_ARGUMENT. {'code': 400, 'message': 'Requests ending with a
model turn are not supported.', 'status': 'INVALID_ARGUMENT'}

Switching Gemini models does not help — the malformed history is what gets replayed. A user interrupt leaves the same state.

Root cause

When a turn dies after text has streamed, engine.py:367 appends a partial assistant message — so the transcript still shows what the user saw — then an error notice. _outbound_messages() drops notices (engine.py:1043), so the partial becomes the final message in the payload, and Engine.retry() (engine.py:246) replays it unmodified. Captured from the running engine rather than read off the source:

history:   user | assistant 'I will request permission to ' | notice/error
outbound:  ['user', 'assistant']        <- notice stripped, partial is the tail

Not Gemini-specific

That was my first assumption and it was wrong. Claude 4.5 and older accept a trailing assistant turn as prefill but 400 when it ends in whitespace — final assistant content cannot end with trailing whitespace — and streamed deltas routinely do; the real cut from my session was 'I will request permission to '. Sonnet 4.6, Opus 4.8 and Fable 5 reject it unconditionally (does not support assistant message prefill). That is three of the four Claude models in the curated picker, with the fourth exposed via whitespace.

Reproduced end-to-end on Sonnet 4.6 through the real engine and retry() against the live API: three retries, three identical 400s.

So the defect is that _outbound_messages() ships a turn that never completed. Guarding one vendor's converter would leave the other models broken.

Fix

Drop the abandoned partial in _outbound_messages(), alongside the notices that function already treats as display-only.
The trailing error/interrupted notice is what identifies it, and that detail is load-bearing: a completed turn's assistant message is structurally identical — trailing, no tool_calls — so a shape-based check silently deletes the last turn of every finished conversation. I wrote that version first, and test_provider_extras_persist_on_message_and_survive_outbound caught it. Trailing tool_calls mean a turn suspended awaiting durable resume, never an abandoned one.

Because notices persist to disk, this also unwedges sessions already broken — which a converter patch would not.

Tests

  • test_retry_after_mid_stream_death_does_not_replay_the_partial — the partial stays in history for the transcript but never reaches the provider
  • test_completed_assistant_turn_is_never_dropped — the counterpart; pins the distinction the notice provides
  • test_consecutive_deaths_do_not_stack_partials_in_the_payload — each failed retry appends its own
  • test_resume_payload_keeps_the_suspended_tool_call — durable resume unaffected; the assistant turn that made the call still reaches the model

Full suite: 1012 passed. The 7 test_bedrock_provider.py failures in my environment are ModuleNotFoundError from a missing boto3 in my venv and fail identically on unmodified main.

Notes

  • Unrelated to fix: repair tool-call/result pairing on load to prevent unrecoverable 400 errors #350 — that repairs tool-call/result pairing in persisted JSONL; this drops an abandoned turn at the outbound boundary. No shared files.
  • Alternative considered: tag the partial at creation and drop by tag. No inference at all, but it only helps future failures, leaving already-wedged sessions on disk broken. Happy to switch if you would rather have the explicit marker.
  • On stock OpenAI a trailing assistant turn is tolerated today, so a retry currently continues from the partial; after this it regenerates the turn. Nothing breaks, but it encodes retry means re-run rather than retry means continue — a call I am happy to have overruled.

A turn that dies mid-stream (provider 503, user interrupt) appends its partial
assistant message to history so the transcript still shows what streamed, then
an error notice. `_outbound_messages()` drops notices, so the partial becomes
the FINAL message sent to the provider, and `retry()` replays it verbatim.

Several providers reject that outright:

  - Gemini: "Requests ending with a model turn are not supported"
  - Claude 4.6+ / Fable 5: "does not support assistant message prefill"
  - Claude 4.5 and older: "final assistant content cannot end with trailing
    whitespace" -- streamed deltas routinely end in a space

So every retry replays the same bad tail and the session is wedged until the
user sends a new message or gives up on it. Reproduced end-to-end against live
APIs on Gemini 3.6 Flash and Claude Sonnet 4.6; three of the four Claude models
in the curated picker reject the payload unconditionally.

Drop the abandoned partial in `_outbound_messages()`, alongside the notices it
already treats as display-only. The trailing error/interrupted notice is what
identifies it: a COMPLETED turn's assistant message is structurally identical
(trailing, no tool_calls) and must be kept, since its `extras` sidecar is
replayed back to the owning provider. Trailing tool_calls mean a turn suspended
awaiting durable resume, never an abandoned one.

Because the notices persist, this also unwedges sessions already broken on disk.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ChakrawarShubham
ChakrawarShubham force-pushed the fixGeminiTrailingModelTurn branch from 14e87b9 to 7edf8b9 Compare July 31, 2026 04:01
@ChakrawarShubham ChakrawarShubham changed the title fix(gemini): guard histories ending on a model turn fix(engine): stop replaying an abandoned partial turn on retry Jul 31, 2026
@ChakrawarShubham

Copy link
Copy Markdown
Author

Since first-time contributor workflows need maintainer approval, I ran CI on my fork so there's something to look at without spending an approval: https://github.com/ChakrawarShubham/openworker/actions/runs/30603081143 — pytest, GUI unit and Playwright e2e all green.

Happy to rework this in whatever direction you prefer. I know engine.py is a file you're actively working in, and the tagging alternative in the notes above may well fit your plans better than what I've done here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Retry replays abandoned partial turn — session permanently wedges with 400 on Gemini, Claude 4.6+, and older Claude

1 participant