Skip to content

feat(codex): track Codex rate-limit windows and show them on the Usage page - #1704

Draft
jzila wants to merge 1 commit into
kenn-io:mainfrom
jzila:codex-rate-limits
Draft

feat(codex): track Codex rate-limit windows and show them on the Usage page#1704
jzila wants to merge 1 commit into
kenn-io:mainfrom
jzila:codex-rate-limits

Conversation

@jzila

@jzila jzila commented Sep 10, 2026

Copy link
Copy Markdown

Preserves Codex rate-limit observations and adds Usage-page cards showing
utilization, reset times, plan, credits, and history.

A vendor-keyed SQLite table backs the current and history endpoints. Indexed
latest-window lookups and SQL downsampling avoid loading full history into Go.
Replay and resync preserve observations without duplicating them.

Codex rollouts lack an account ID, so their cards group by machine. Rate-limit data
is SQLite-only. The data-version bump causes a source reparse on the next sync to
backfill existing history.

Review internal/db/rate_limit_snapshots.go, the Codex parser and sync paths, and
frontend/src/lib/stores/ratelimits.svelte.ts.

@jzila
jzila marked this pull request as draft September 10, 2026 18:11
@roborev-ci

roborev-ci Bot commented Sep 10, 2026

Copy link
Copy Markdown

roborev: Combined Review (0e5d652)

Verdict: Changes require fixes for 4 findings.

High

  • internal/sync/s3.go:43-69; internal/sync/engine.go:19839-19845: S3 ID prefixing updates the parsed session ID but not RateLimitSnapshots, so snapshots retain the native Codex session ID and foreign-key insertion fails for S3 Codex sessions containing rate-limit data. Apply the prefix to snapshot SessionID values too, or overwrite them with the final stored session ID before insertion and recompute deduplication keys.

    Reported by: codex

Medium

  • internal/db/codex_rate_limit_snapshots.go:229-235,264-275; frontend/src/lib/stores/ratelimits.svelte.ts:81-84: The Usage machine filter supports comma-separated selections, but rate-limit queries use exact equality, so selecting multiple machines returns no snapshots. Split machine filters and use the same IN-predicate behavior as the other session and usage queries.

    Reported by: codex

  • frontend/src/lib/components/ratelimits/RateLimitsSection.svelte:17-20: History bounds use UTC midnight through 23:59:59Z even though Usage dates are browser-local calendar dates; this produces wrong days for non-UTC users, excludes sub-second observations in the final second, and ignores the narrower selected time range after a chart brush. Use the effective Usage range, convert local calendar boundaries to UTC or pass timezone information to the endpoint, and use an exclusive next-day upper bound or normalize timestamps chronologically.

    Reported by: codex

  • internal/db/codex_rate_limit_snapshots.go:261-274; internal/service/rate_limits.go:104-110: The history endpoint accepts offset-bearing RFC3339 timestamps but compares raw request strings lexicographically with stored UTC strings, causing observations to be included or excluded at the wrong instants. Parse RFC3339Nano inputs and normalize them to UTC before querying, or compare parsed/unix timestamp values in SQLite.

    Reported by: codex


Reviewers: 2 done | Synthesis: codex, 15s | Total: 19m38s

@roborev-ci

roborev-ci Bot commented Sep 10, 2026

Copy link
Copy Markdown

roborev: Combined Review (d75459e)

Verdict: Changes require fixes for 4 findings.

High

  • internal/sync/engine.go:19832-19863: Rate-limit snapshots retain the raw codex: session ID during remote or ID-prefixed syncs, while the stored session uses the prefixed ID; the foreign-key insert can fail or attach to an unintended unprefixed session. Use the final stored sessionID as the snapshot SessionID, or apply the same ID prefix during snapshot rewriting; add coverage for remote-prefixed syncs.

    Reported by: codex

Medium

  • internal/db/codex_rate_limit_snapshots.go:228-230: The shared machine filter supports multiple comma-separated machines, but the current-snapshot SQL compares the entire string with machine = ?. Selecting multiple machines makes the rate-limit section incorrectly empty. Parse the machine filter into an IN predicate using the same semantics as the other store queries, or fetch each selected machine separately.

    Reported by: codex

  • frontend/src/lib/components/ratelimits/RateLimitsSection.svelte:32-40: The rate-limit section ignores sessions.filters.agent; selecting a non-Codex agent on the Usage page still displays Codex rate-limit cards because only the separate exclusion list is checked. Pass the shared agent filter to fetchCurrent, react to agent changes, and hide the section when the selected agents exclude Codex.

    Reported by: codex

  • frontend/src/lib/components/ratelimits/RateLimitsSection.svelte:19-20: History bounds are hard-coded to UTC and use an inclusive 23:59:59Z endpoint, which does not match the Usage page's local-time date range and excludes observations with fractional seconds during the final second. Convert local calendar boundaries with the browser timezone/DST rules and use an exclusive next-day upper bound, or preserve nanosecond precision through the end of the selected day.

    Reported by: codex


Reviewers: 2 done | Synthesis: codex, 11s | Total: 13m42s

@roborev-ci

roborev-ci Bot commented Sep 10, 2026

Copy link
Copy Markdown

roborev: Combined Review (25eabcd)

Verdict: Changes require fixes for 1 finding.

High

  • internal/db/orphaned.go:1919: Full resync preservation omits the new codex_rate_limit_snapshots table, so snapshots for orphaned or trashed sessions—and detached rows whose session was deleted and set to NULL—can be lost during resync. Extend resync copy logic with a backward-compatible, deduplicating copy of codex_rate_limit_snapshots for session-linked orphan/trashed rows and detached rows, guarded by a source-table existence check.

    Reported by: codex


Reviewers: 2 done | Synthesis: codex, 7s | Total: 11m35s

@roborev-ci

roborev-ci Bot commented Sep 10, 2026

Copy link
Copy Markdown

roborev: Combined Review (7772c79)

Verdict: Changes require fixes for 1 finding.

High

  • internal/sync/engine.go:3498; internal/db/rate_limit_snapshots.go:509-524: Resync copies rate-limit rows before orphaned sessions are restored. Because session_id is a foreign key, snapshots for orphaned or parser-excluded sessions can make the bulk INSERT OR IGNORE fail, discarding all copied rate-limit history. Copy orphan sessions before snapshots, and/or set session_id to NULL for source rows without a matching destination session so every valid snapshot is preserved.

    Reported by: codex


Reviewers: 2 done | Synthesis: codex, 8s | Total: 7m24s

@jzila
jzila force-pushed the codex-rate-limits branch 2 times, most recently from 40ba274 to 6e23b7f Compare September 10, 2026 21:50
@roborev-ci

roborev-ci Bot commented Sep 10, 2026

Copy link
Copy Markdown

roborev: Combined Review (6e23b7f)

Verdict: Changes require fixes for 1 finding.

Medium

  • frontend/src/lib/components/ratelimits/RateLimitCard.svelte:68-76; frontend/src/lib/utils/rateLimitFormat.ts:52-58: Valid resetsAt: 0 timestamps are treated as unknown because the card requires resetsAt > 0 and the formatter rejects non-positive timestamps, contradicting the backend/API distinction between null and the Unix epoch. Treat any finite numeric resetsAt as known, including zero; return null from the countdown only after the reset has passed, and add a frontend test for resetsAt: 0.

    Reported by: codex


Reviewers: 2 done | Synthesis: codex, 8s | Total: 8m52s

@jzila
jzila force-pushed the codex-rate-limits branch 8 times, most recently from ed4a86a to 64f4371 Compare September 11, 2026 00:42
@roborev-ci

roborev-ci Bot commented Sep 11, 2026

Copy link
Copy Markdown

roborev: Combined Review (64f4371)

Verdict: Changes require fixes for 2 findings.

Medium

  • internal/db/rate_limit_snapshots.go:214-226: Rate-limit deduplication uses only session ID, timestamp, limit ID, and window kind, so multiple token_count events sharing a timestamp can cause later observations with changed usage to be discarded. Carry a stable per-event discriminator from parsing, such as a source line, byte offset, or sequence, and include it in the deduplication key while preserving reparse stability.

    Reported by: codex

  • internal/db/rate_limit_snapshots.go:522-555; frontend/src/lib/stores/ratelimits.svelte.ts:216-250: History is downsampled before the frontend filters by plan type. When plan changes cause multiple plan series to share the 200-point bucket budget, a card can lose observations or show no chart despite having history. Add plan_type to the history filter/API, apply it before downsampling, and pass the card's plan type with the history request.

    Reported by: codex


Reviewers: 2 done | Synthesis: codex, 8s | Total: 15m21s

@roborev-ci

roborev-ci Bot commented Sep 11, 2026

Copy link
Copy Markdown

roborev: Combined Review (ffd2681)

Verdict: Changes require fixes for 2 findings.

Medium

  • internal/parser/codex.go:588; internal/parser/traex.go: The shared Codex-format parser records rate-limit snapshots for TraeX, while the write path leaves Vendor empty and the database defaults it to "codex," causing TraeX payloads to be displayed and filtered as Codex. Gate extraction and persistence on the concrete agent, or propagate and relabel the actual vendor for every snapshot; add a TraeX fixture with a non-null rate_limits payload.

    Reported by: codex

  • internal/db/rate_limit_snapshots.go:468: Latest snapshots are partitioned by plan_type even though plan_type is an unstable label, allowing newer observations to coexist with stale windows from an older plan bucket and produce duplicate or outdated cards. Rank current data by the stable vendor/machine/account/limit observation bucket without plan_type, then return all windows from that newest observation. Keep plan_type only as display data and exclude it from frontend keyed identities.

    Reported by: codex


Reviewers: 2 done | Synthesis: codex, 9s | Total: 10m14s

@roborev-ci

roborev-ci Bot commented Sep 11, 2026

Copy link
Copy Markdown

roborev: Combined Review (7d16330)

Verdict: Changes require fixes for 2 findings.

Medium

  • internal/db/rate_limit_snapshots.go:420: The new endpoints unconditionally query rate_limit_snapshots, but OpenReadOnly accepts older otherwise-compatible archives without creating or migrating this table, causing current or history requests to fail with no such table. Treat an absent optional table as an empty result or reject the archive through the existing schema-upgrade path, and add coverage for older read-only archives.

    Reported by: codex

  • internal/db/rate_limit_snapshots.go:240: Rate-limit rows are append-only for fallback Codex parses marked DataVersionNeedsRetry. Replayed parent token_count observations can remain after a later authoritative parse resolves the parent and replaces messages, producing false history and potentially affecting current state. Delete and replace the session's snapshots when replacing a retried parse, or otherwise reconcile fallback observations before inserting authoritative data.

    Reported by: codex


Reviewers: 2 done | Synthesis: codex, 8s | Total: 14m9s

@roborev-ci

roborev-ci Bot commented Sep 11, 2026

Copy link
Copy Markdown

roborev: Combined Review (df56d65)

Verdict: Changes require fixes for 2 findings.

Medium

  • internal/db/rate_limit_snapshots.go:530-534, 702-705: The API documents account_id as ignored for Codex rows, but current and history queries unconditionally add account_id = ?. Because Codex rows store an empty account ID, any nonempty account_id request returns no Codex data. Ignore AccountID when the selected vendor is Codex, or explicitly reject it instead of silently filtering out all rows.

    Reported by: codex

  • internal/db/rate_limit_snapshots.go:887-909: Resync rebuilds current sessions and deletes their old rate-limit rows, then CopyRateLimitSnapshotsFrom copies every old row back whenever the session ID still exists in the rebuilt archive. This can resurrect observations from superseded fallbacks or rewritten transcripts, producing stale current values and history entries. During resync, copy snapshots only for orphaned or unrestored sessions, while preserving rows whose source sessions are intentionally retained without a reparse.

    Reported by: codex


Reviewers: 2 done | Synthesis: codex, 10s | Total: 9m37s

@roborev-ci

roborev-ci Bot commented Sep 11, 2026

Copy link
Copy Markdown

roborev: Combined Review (e0a44e8)

Verdict: No findings at or above medium severity.


Reviewers: 2 done | Synthesis: codex, 7s | Total: 11m39s

@jzila
jzila marked this pull request as ready for review September 11, 2026 03:57
@roborev-ci

roborev-ci Bot commented Sep 11, 2026

Copy link
Copy Markdown

roborev: Combined Review (54381f9)

Verdict: Changes require fixes for 1 finding.

Medium

  • internal/db/rate_limit_snapshots.go:490: sync.Once is marked complete even when the sqlite_master probe returns an error, permanently leaving rateLimitSnapshotsTableFound false and causing current and history endpoints to return empty results for the lifetime of the DB instance. Use retryable initialization state or a mutex, and cache the result only after a successful probe.

    Reported by: codex


Reviewers: 2 done | Synthesis: codex, 7s | Total: 17m29s

@roborev-ci

roborev-ci Bot commented Sep 11, 2026

Copy link
Copy Markdown

roborev: Combined Review (46ee3b2)

Verdict: Changes require fixes for 1 finding.

Medium

  • internal/sync/engine.go:3543: ResyncAll restores trashed sessions before parsing but copies rate-limit snapshots using only orphaned IDs, so snapshots belonging to trashed sessions already present in the replacement database are excluded and their history is lost. Include IDs returned by CopyTrashedDataFrom when copying rate-limit snapshots, or copy those snapshots during trashed-session restoration; add a resync test for a trashed session.

    Reported by: codex


Reviewers: 2 done | Synthesis: codex, 7s | Total: 16m24s

@roborev-ci

roborev-ci Bot commented Sep 11, 2026

Copy link
Copy Markdown

roborev: Combined Review (7ce17bb)

Verdict: Changes require fixes for 1 finding.

Medium

  • internal/db/rate_limit_snapshots.go:414, 933-961: Resync copies snapshots for sessions missing from the destination with session_id=NULL while preserving their dedup_key. If that session is later re-imported, INSERT OR IGNORE conflicts with the detached row, discards the new attached snapshot, and leaves stale data outside session replacement/deletion. Handle deduplication conflicts by reattaching or updating a matching NULL-session row when the incoming session ID is valid, or preserve a separate source-session key for reattachment.

    Reported by: codex


Reviewers: 2 done | Synthesis: codex, 8s | Total: 18m39s

@jzila
jzila marked this pull request as draft September 11, 2026 06:36
…e page

Codex CLI writes a `rate_limits` object into its `token_count` events
(up to two windows, a short `primary` window and a longer `secondary`
one, e.g. 5 hours and 7 days) alongside the plan type and credit
balance. agentsview now extracts that object during Codex parsing,
persists each observation, and surfaces it on the Usage page as a
"Rate limits" section: one card per window showing used percent, time
to reset, plan type, credit balance, and a small history chart of
used percent over the selected date range.

Schema and identity

Observations are stored in a new `rate_limit_snapshots` table
(SQLite only, alongside cursor_usage_events and the Codex
incremental-import tables -- out of scope for the SQLite/PostgreSQL/
DuckDB parity rule). The table is vendor-keyed (`vendor`, `'codex'`
for every row today) from the start rather than Codex-specific,
because a second PR building on this one adds a Claude rate-limit
poller that needs the same table: shipping the vendor-neutral shape
now means that PR only adds rows, instead of carrying a migration
that renames the table and backfills a `vendor` column for a table
that had not even shipped to users yet. `account_id`, `account_label`,
`scope_label`, and `details` are reserved the same way for a vendor
whose rate-limit source has that shape, and stay empty on every Codex
row.

Codex rollouts carry no stable per-account identifier -- no account
id, user id, email, or org field appears in session_meta or
token_count payloads (see docs/internal/session-format-sources.md) --
so a Codex snapshot's identity is (machine, limit_id, plan_type,
window_kind) rather than an account. Rows are written with INSERT OR
IGNORE against a unique dedup_key (source session id + observed
timestamp + limit id + window kind), never a delete-then-reinsert, so
both a full parse and an incremental parse converge on the same rows
without duplicating or losing history. session_id is nullable (ON
DELETE SET NULL) so a row survives its source session being deleted.
resets_at is also nullable end to end -- the parser type, the table,
and the API response all preserve a genuinely unknown reset time as
null rather than flattening it to 0, which previously would have
shown as a bogus countdown to the unix epoch.

Because this backfills history from existing rollout files, the
parser data version is bumped (to 108) so an archive written by an
older binary gets a full reparse and picks up the rate_limits history
its rollout files already contained, instead of silently staying
empty until the next observation.

The "current" query (LatestRateLimitSnapshots) resolves "latest" per
(vendor, machine, account_id, limit_id, plan_type) bucket, one level
above window_kind, and returns every window belonging to that bucket's
single newest observation, rather than ranking each window_kind
independently: the latter let a window that stopped being reported
(e.g. a session moving from primary+secondary windows to primary-only)
keep showing its last-known row indefinitely. The winning observation
is further disambiguated by a persisted observation_key column, stored
once at insert time from the then-current session_id and independent
of the nullable session_id foreign key: two different sessions on one
machine and account can legitimately report the same limit/plan at the
exact same instant (Codex's rate limit is account-wide), and using
session_id itself for this at query time broke either when two such
sessions were both deleted (their now-NULL session_ids would compare
equal again) or, if that NULL case was special-cased per row instead,
when one deleted session's own primary+secondary windows could no
longer be told apart from each other. A key fixed at write time and
never recomputed avoids both failure modes.

The history endpoint downsamples a wide date range instead of
returning every matching observation: `max_points` (default 500) caps
the response, dividing the range into that many equal-width time
buckets and keeping only the most recently observed row per bucket, so
a long-lived window's full history does not grow the response (or
RateLimitHistoryChart's point count) unboundedly. The chart itself
requests a smaller point budget sized for its own sparkline-sized
rendering.

API and frontend

New routes `GET /api/v1/rate-limits/current` and `.../history` accept
`vendor`, `account_id`, and `machine` filters (`agent` is kept as a
deprecated alias for `vendor`, matching the shared session filter's
comma-separated selection semantics via
`RateLimitAgentMatchesVendor`); `history` additionally accepts
`limit_id`, `window`, `since`, `until`, and `max_points`. PostgreSQL and
DuckDB implement the read side as no-ops, so the section is simply
hidden when either backend is the active read store.

The frontend groups cards by vendor and then by account; since Codex
reports no account identity, its account group is keyed by machine
instead, so two machines syncing the same account still render as two
groups. A window with no known reset time shows "Reset time unknown"
instead of a countdown. The history chart now plots observed time on
a real time scale (it previously used a categorical point scale that
spaced every observation evenly by index, which visually flattened
bursts and gaps in the actual observation cadence).

Robustness fixes folded in from post-implementation review

- A full resync now copies existing rate_limit_snapshots rows into
  the replacement archive, the same way model pricing is copied, so
  previously observed windows survive the swap instead of going empty
  until the next observation. This copy runs after orphaned sessions
  are restored, not before: session_id is a foreign key, and a
  snapshot belonging to an orphaned session would otherwise violate
  it during the copy (INSERT OR IGNORE does not suppress a
  foreign-key violation the way it suppresses a duplicate), silently
  losing every row in the table rather than just that session's. The
  copy also NULLs session_id for any row whose session still does not
  exist in the destination even after orphan restoration -- one
  superseded by a reparse under a different id, or one excluded as
  parser-excluded -- rather than copying that id unchanged and hitting
  the same constraint; dedup_key and observation_key are preserved
  from the source unchanged either way. This history cannot be
  reconstructed once lost, so a failed copy aborts the swap instead of
  merely warning.
- A rate_limits payload missing limit_id is skipped rather than
  failing the whole write -- since this write is normally part of a
  larger session write, the old behavior could fail an entire
  session's ingestion over one malformed rate-limit entry.
- The machine and agent/vendor filters use the same comma-separated
  IN/contains semantics as the rest of the shared session filters,
  and rate-limit history queries parse and normalize since/until as
  RFC3339Nano rather than comparing raw request strings against the
  stored UTC column.
- The incremental-parse cursor seed (a prefix scan that reconstructs
  cursor state without ever returning a session result) no longer
  accumulates rate-limit observations while scanning: they were never
  read out of that path, so collecting them grew memory with the
  whole scanned prefix on every incremental-parse cache miss instead
  of staying bounded.

docs/agents/storage.md and docs/token-usage.md are updated to
describe the new table and endpoints.
@roborev-ci

roborev-ci Bot commented Sep 11, 2026

Copy link
Copy Markdown

roborev: Combined Review (7fe198b)

Verdict: Changes require fixes for 1 finding.

High

  • internal/db/rate_limit_snapshots.go:985: Rate-limit resync copies snapshots for session IDs absent from the rebuilt database even when those sessions were superseded by newer sessions for the same file path. Detached rows can duplicate fresh snapshots and win latest-row tie-breaking due to higher IDs, exposing obsolete usage values. Apply the same file-path replacement exclusion logic used for orphan-data copying, or pass an explicit exclusion set from session resync and omit stale snapshots instead of detaching them.

    Reported by: codex


Reviewers: 2 done | Synthesis: codex, 9s | Total: 8m18s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant