Stream reasoning tokens into interactive CLI #11
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…-reasoning-tokens-to-cli
- Gateway now intercepts stream/request messages and responds with stream/open - Generates unique stream IDs and tracks active streams per space - Handles stream data frames in #streamID#data format - Cleans up streams on stream/close messages - Added sendStreamData method to MEWParticipant for sending stream frames - Updated spec to clarify gateway's role in stream management 🤖 Generated with Claude Code Co-Authored-By: Claude <[email protected]>
- Stream token count progress during reasoning for visual feedback - Add /streams command to toggle display of raw stream data frames - Update handleStreamData to process stream frames and update reasoning bar - Fix requestStream to return message ID for correlation - Update reasoning bar to show 'X tokens' instead of '0 thoughts' during streaming - Add stream_options to OpenAI calls for better progress tracking Even though OpenAI reasoning models don't stream actual reasoning text, this provides real-time feedback that the model is working. 🤖 Generated with Claude Code Co-Authored-By: Claude <[email protected]>
- Add postbuild script to ensure mew-agent executable permissions - Add chat/acknowledge, chat/cancel capabilities to agent templates - Add stream capabilities (stream/request, stream/open, stream/close) to templates - Update test scenario space configs with new capabilities These capabilities are required for proper chat acknowledgment and streaming support. 🤖 Generated with Claude Code Co-Authored-By: Claude <[email protected]>
- Remove test/.mew and spaces/.mew workspace entries - These are runtime directories that shouldn't be tracked - Mark some dependencies as dev dependencies 🤖 Generated with Claude Code Co-Authored-By: Claude <[email protected]>
- Replace justifyContent='space-between' with individual spacing - Display elapsed time and token count separately with marginLeft - Show action text without redundant 'Using' prefix - Track and display action field from reasoning/thought messages - Better horizontal spacing for improved readability 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
UI improvements: - Add proper spacing around elapsed seconds and token counts - Hide '0 thoughts' display when reasoning starts - Increase margins for better visual separation - Add inline padding around seconds value Documentation: - Add comprehensive reasoning bar section to CLI spec - Document fixed-height layout strategy - Explain token streaming vs thought counting modes - Detail UI jitter prevention techniques 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Filter out stream/data in handleIncomingMessage - Remove addMessage call in handleStreamData function - Stream data only appears in Signal Board's Last Frames section - Prevents message list flooding during active streaming - Keeps message count accurate by excluding stream frames 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Merge /reason-cancel functionality into /cancel command - /cancel now cancels active reasoning if present - Falls back to cancelling pending chat entries if no reasoning - Update reasoning bar hint to show '/cancel' instead of '/reason-cancel' - Update help text to reflect dual purpose - Remove separate 'Reasoning' section from help modal 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Import useCallback from React - Throttle reasoning state updates to 100ms intervals - Store pending updates in ref to batch rapid token count changes - Clean up throttle timer on component unmount - Significantly improves input responsiveness during active streaming - Prevents UI re-renders on every token, reducing lag 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- Check reasoningCancelled flag during stream processing - Immediately abort OpenAI stream when cancelled - Return special 'cancelled' action from reason() method - Handle 'cancelled' action in act() method - Send reasoning/conclusion immediately on cancel event - Clear reasoning context after cancellation - Prevents agent from continuing in background after cancel 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
- OpenAI SDK doesn't support 'signal' parameter - Remove AbortController usage entirely - Still check reasoningCancelled flag to exit stream loop early - Note: Can't actually abort underlying HTTP request with OpenAI SDK - Fixes streaming that was broken with 400 error 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
The MEWAgent was not sending stream/close messages after finishing with a stream, causing streams to accumulate in the Signal Board. Now properly sends stream/close when reasoning concludes or is cancelled. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Walkthrough
Start with the MEWAgent streaming handshake.
Look at the new
reasoningStreamInfoworkflow that captures pending chunks, wires upstream/request→stream/opencorrelations, and tears everything down onstream/closeorreasoning/cancel. This section shows how each MEW stream event is intercepted so the agent can buffer reasoning text until a stream is negotiated, then flush it once the stream ID is known.Review the streaming inference path.
The agent now prefers
createStreamingReasoning, which subscribes to OpenAI’s stream iterator, emits incremental token chunks, and still aggregates tool-call deltas before returning. Immediately after, the ReAct loop records whether streaming succeeded so it can fall back to the legacy single-shot call and still mirror the final text into the stream path.Move to the Ink CLI’s WebSocket plumbing.
At the top of
advanced-interactive-ui.js, note the new refs and state for active reasoning sessions and MEW streams. ThehandleStreamFramehelper is the key addition: it decodes the chunk metadata, keeps activity timestamps, and merges reasoning tokens into the active session instead of printing raw frames.Trace how regular envelopes update reasoning state.
In
handleIncomingMessage, follow the new branches that mapreasoning/start,reasoning/thought,reasoning/conclusion, and the supportingstream/*envelopes into the same session object tracked by the CLI. This section is where stream requests are matched to context IDs so the UI knows which stream belongs to which reasoning turn.Finish with the UI surfaces.
ReasoningStatusrenders the live spinner, streamed text preview, and token metrics;MessageDisplay/ReasoningDisplaygive reasoning messages their own formatting; and the status bar now includes stream/context usage cues so the reader can see the impact of the new telemetry while scanning the footer.Testing
https://chatgpt.com/codex/tasks/task_e_68d122abafe48325ae624de59a3910e6