feat: show token usage and rate-limit footer per turn - #111
Merged
shauryagangrade merged 2 commits intoAug 24, 2026
Conversation
Fixes shauryagangrade#70 Free OpenRouter models are rate-limited and context consumption is invisible. ChatOpenAI streams carry usage metadata but GCode dropped it. Capture usage from response.usage_metadata and response_metadata.token_usage/usage, plus X-RateLimit-Remaining headers when present, and print a compact footer after each turn: tokens in/out, total, approximate cost, and remaining rate limit. Silently omits when provider supplies no usage. Rate-limit remaining is surfaced before a 429, matching the acceptance criteria. Validation: py_compile passes, git diff --check clean; footer prints only when usage present, never breaks the turn.
_stream previously rebuilt the AIMessage without usage_metadata / response_metadata, discarding the provider's token counts before _print_usage ever saw them. Also drop the rate-limit header parsing (ChatOpenAI never exposes response headers), keep zero token counts via explicit None checks, and add unit tests.
shauryagangrade
marked this pull request as ready for review
August 24, 2026 16:07
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #70
Problem
Free OpenRouter models are rate-limited and users get no feedback on how much context they're consuming.
ChatOpenAIstreams carry usage metadata, but GCode drops it. Issue #70 proposes capturingusage/rate-limit info from stream chunks and printing a compact footer after each turn: tokens in/out, approximate cost, and remaining rate limit, plus a/usagesurface.Change
gcode/agent.py:_print_usage(response, ui)that extracts usage fromresponse.usage_metadata(LangChain 0.3+) with fallback toresponse.response_metadata['token_usage'/'usage']_streamto carryusage_metadata/response_metadataonto the canonicalAIMessageit returns — without this the rebuilt message discarded the provider's token counts and the footer never printed_print_usageafter each_streaminrun_turn(initial + each tool-loop iteration, so multi-tool turns show one line per model call); silently omits when provider supplies no usageUsage: <in> in / <out> out · total <n> · ~$<cost>(only present fields shown; zero counts are preserved via explicit None checks)ChatOpenAInever exposes raw response headers inresponse_metadata, so that path was unreachable. Surfacing rate-limit state needs an httpx hook on the underlying client — tracked as a follow-upWhy this approach
Reuses the existing streaming path; the footer is the minimal user-visible fix matching the acceptance criterion "A turn prints usage when the provider supplies it; silently omits otherwise." No new
/usagecommand needed because the footer is automatic; folding into/historywould be redundant.Testing
Unit tests in
tests/test_agent.py:_streampreservesusage_metadata; footer renders fromusage_metadata, fromresponse_metadata['token_usage'], keeps zero counts, shows cost, and stays silent with no usage.Manual (with API key): start a session, trigger a turn, and observe footer like
Usage: 842 in / 128 out · total 970when OpenRouter returns usage; no footer when Ollama/local returns no usage.Documentation and release impact
Review notes