feat(core,llm,memory): add per-message usage/cost tracking#6628
Merged
Conversation
github-actions
Bot
added
enhancement
bug-ops
enabled auto-merge (squash)
July 21, 2026 00:46
bug-ops
force-pushed
the
feat/issue-6549/per-message-usage-cost-tracking
branch
2 times, most recently
from
July 21, 2026 00:57
2b60f79 to
87d4eb7
Compare
Every LLM call that feeds CostTracker::record_usage now also emits a durable usage_records row (migration 115, SQLite + PostgreSQL): input/ output/cache tokens, cost in cents, latency, TTFT/TTFB, and tokens/sec, queryable independently of the running session/daily aggregate. Conversational turns link message_id to their persisted assistant message; planner, aggregator, and verifier-ensemble calls carry message_id = NULL since they produce no conversational message. Every row is priced via CostTracker::price_of, the same formula the daily aggregate uses, so current-day SUM(cost_cents) reconciles with CostTracker.current_spend(). ttft_ms is true time-to-first-token on the one production streaming path (speculative decoding, captured at SpeculativeStreamDrainer's stream-consumption point) and a time-to-first-byte proxy elsewhere, measured per-attempt inside the shared retry helper so a retried call is never inflated by backoff sleep. AnyProvider/MaskedProvider/ TriageRouter delegate last_ttft_ms to the active inner provider. Usage-row writes are inline-awaited on the same call sites that already update CostTracker, gated on the existing [cost] enabled switch - no new tokio::spawn, no new config surface. Closes #6549
bug-ops
force-pushed
the
feat/issue-6549/per-message-usage-cost-tracking
branch
from
July 21, 2026 01:01
87d4eb7 to
348c0ff
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
usage_recordstable (migration 115, SQLite + PostgreSQL) that records a per-LLM-call usage ledger (tokens, cost, latency, TTFT/TTFB, tokens/sec), additive toCostTracker's existing provider-keyed daily aggregate.CostTracker::record_usagealso writes a paired row: the turn loop (message_id-linked to the persisted assistant message), the orchestration planner and aggregator, and verifier-ensemble members (the latter three carrymessage_id = NULL, since no persisted conversational message exists for them). Current-daySUM(cost_cents)reconciles withCostTracker.current_spend().ttft_msis true time-to-first-token on the one production streaming path (speculative decoding, captured atSpeculativeStreamDrainer::drive's stream-consumption point) and a time-to-first-byte proxy elsewhere, measured per-attempt inside the shared retry helper so a retried call is never inflated by backoff sleep.NULLonly for the in-process Candle backend and forRouterProvider(a pre-existing, unrelated usage-propagation gap, tracked as a follow-up, not fixed here).[cost] enabledgate and are inline-awaited on the same call sites that already updateCostTracker; no newtokio::spawn.Closes #6549
Test plan
cargo +nightly fmt --checkcargo clippy --profile ci --workspace --all-targets --features "desktop,ide,server,chat,pdf,scheduler,testing" -- -D warningscargo nextest run --config-file .github/nextest.toml --workspace --features "desktop,ide,server,chat,pdf,scheduler" --lib --bins— 15041 passed, 0 failedRUSTFLAGS="-D warnings" RUSTDOCFLAGS="--deny rustdoc::broken_intra_doc_links" cargo doc --no-deps --workspace --features "desktop,ide,server,chat,pdf,scheduler"cargo test --doc --workspace --features "desktop,ide,server,chat,pdf,scheduler"gitleaks protect --staged --no-banner --redact— no leaksAnyProvider/MaskedProviderTTFT delegation, retry TTFB-not-inflated-by-backoff, pending-usage no-leak-on-filtered-persist,price_offallback parity, reconciliation invariant (turn + planner), disabled-cost-tracker gating.local/testing/playbooks/per-message-usage-cost-tracking.md