Skip to content

feat(ai): SuperGrok usage tracking via billing API#4874

Merged
can1357 merged 10 commits into
can1357:mainfrom
any-victor:feat/xai-oauth-usage
Jul 23, 2026
Merged

feat(ai): SuperGrok usage tracking via billing API#4874
can1357 merged 10 commits into
can1357:mainfrom
any-victor:feat/xai-oauth-usage

Conversation

@any-victor

@any-victor any-victor commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds an xai-oauth UsageProvider that polls SuperGrok subscription usage from GET https://cli-chat-proxy.grok.com/v1/billing?format=credits.
  • Collects stored OAuth credentials first; uses XAI_OAUTH_TOKEN only when no usable stored OAuth credential exists. Shared XAI_API_KEY credentials are never sent to the SuperGrok billing host (supports() is OAuth-only; AuthStorage skips api_key material for this provider).
  • Billing headers include Authorization: Bearer … plus the official CLI gate header X-XAI-Token-Auth: xai-grok-cli (same host fingerprint as Grok Build CLI traffic).
  • Maps weekly credit and per-product limits (GrokBuild, Api, and any other product that reports a percent) plus a positive on-demand cap; product rows without usagePercent (e.g. live GrokChat) are skipped. Enriches usage metadata from OIDC userinfo when an older credential lacks an email.
  • Rejects redirects for every bearer-, device-code-, or refresh-token-authenticated xAI request, preventing credentials from being forwarded to an unvalidated host.
  • Pins billing URLs to HTTPS *.grok.com (separate from OIDC *.x.ai host validation).
  • Keeps recently-ended weekly periods visible so /usage does not blank across rollover.

Fixes #5065

Not in this PR

Test plan

  • bun test packages/ai/test/xai-oauth-usage.test.ts packages/ai/test/auth-storage-xai-oauth-usage.test.ts packages/ai/src/registry/oauth/__tests__/xai-oauth.test.ts
  • Rebased onto latest main (17.0.5)
  • Changelog under Unreleased
  • API-key-only env never probes SuperGrok billing
  • Live SuperGrok OAuth verification (community): billing 200 + omp usage --provider xai-oauth --json report shape

@github-actions github-actions Bot added the vouched Passed the vouch gate label Jul 9, 2026
@roboomp roboomp added auth Authentication and login flows feat provider:xai Provider-specific issues for xai providers LLM provider-specific issues review:p2 triaged labels Jul 9, 2026
@roboomp

roboomp commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Review fallback: GitHub rejected the batched inline review after staged comments included stale line positions, and the review staging buffer is not clearable from this read-only session. Concrete findings:

  • blockingpackages/ai/src/usage/xai-oauth.ts:283-285 accepts every api_key credential for xai-oauth. Because packages/ai/src/auth-storage.ts:544 adds this provider to the default usage fanout, AuthStorage.#collectUsageRequests() will synthesize an env credential via getEnvApiKey("xai-oauth") when no stored credentials exist. The catalog maps xai-oauth to XAI_OAUTH_TOKEN || XAI_API_KEY, so users with only a normal XAI_API_KEY will now run the xAI OAuth userinfo probe plus Grok billing probe and send that API key to cli-chat-proxy.grok.com despite never configuring SuperGrok OAuth. Gate api-key usage to a dedicated OAuth bearer source (XAI_OAUTH_TOKEN, JWT-shaped bearer, or stored account identity) rather than the shared fallback.
  • should-fixpackages/ai/src/auth-storage.ts:544 registers a new default usage provider, but packages/ai/CHANGELOG.md:3 has an empty ## [Unreleased] section. Repo convention requires a changelog entry for this user-visible omp usage / status-line behavior change.

Rank applied: review:p2 because this is a new provider-facing default behavior that needs maintainer acceptance after the integration fix above.

Verification: bun test packages/ai/test/xai-oauth-usage.test.ts and bun test packages/ai/test/provider-registry.test.ts passed in the checked-out worktree. Thanks for the focused tests and HAR-backed wire mapping.

@any-victor
any-victor force-pushed the feat/xai-oauth-usage branch 3 times, most recently from dd827a3 to b5213ea Compare July 10, 2026 00:06
@ART1KZ

ART1KZ commented Jul 10, 2026

Copy link
Copy Markdown

Thanks for this — this is the right fix for #5065 on the core xai-oauth credential path.

Quick scope note so we don’t double-track:

Layer What #4874 covers Still separate
Usage UI xai-oauth → CLI billing → SuperGrok weekly + products including GrokBuild % credentials stored under a future/extension provider id grok-build won’t hit supports() here
Chat product path not this PR Build inference still needs cli-chat-proxy routing (#4945), not only seeing GrokBuild in /usage

So: don’t treat this as “Grok Build chat is done” — this is quota visibility for SuperGrok OAuth. That’s still very useful and should land.

If useful as a tiny follow-up after merge: allow the same billing UsageProvider for provider id grok-build (same OAuth token family / same billing host), so Build-extension credentials also appear in /usage without a second implementation.

Prototype extension for the Build chat path (not competing with this usage PR): https://github.com/ART1KZ/omp-grok-build
Design discussion: #5092

@any-victor
any-victor force-pushed the feat/xai-oauth-usage branch 3 times, most recently from 40eabd5 to b92246b Compare July 10, 2026 22:21
@IntegralMedia01

Copy link
Copy Markdown

any updates on this one ? :)

@any-victor

Copy link
Copy Markdown
Contributor Author

any updates on this one ? :)

@IntegralMedia01 I don't think this would make sense anymore, since the xAI provider on OMP draws from API usage instead of the subscription quota and there are no plans for supporting this.

@metaphorics mentioned he would implement Grok Build provider as an extension. I have the full chain of requests from grok-cli captured locally though, so I may publish this as a plugin for full parity once I have some spare time...

PS: oh, here it is, https://github.com/metaphorics/oh-my-pi-plugin-grok-build, I'm gonna check it out as well.

However, as of now there's no clean way of adding new usage providers via extensions, so I may open a PR here to address this. And then, also open a PR for @metaphorics plugin adding the usage provider.

@bnivanov

Copy link
Copy Markdown

Live verification of this PR against a real SuperGrok OAuth account — I hit every symptom from #5065 (xai-oauth invisible in omp usage) and this branch fixes it end to end.

Environment: macOS arm64, branch built from source (b92246b3), credential = stored xai-oauth OAuth token (SuperGrok).

  1. Endpoint probeGET https://cli-chat-proxy.grok.com/v1/billing?format=credits with Authorization: Bearer <token> + X-XAI-Token-Auth: xai-grok-cliHTTP 200:
    • currentPeriod.type = USAGE_PERIOD_TYPE_WEEKLY, startend exactly 7 days apart
    • creditUsagePercent: 34.0
    • productUsage: [{ product: "Api", usagePercent: 34.0 }, { product: "GrokChat" }] — note GrokChat arrives without a percent; the parser skips it cleanly
    • onDemandCap.val = 0 → on-demand limit correctly omitted
  2. omp usage --provider xai-oauth --json on this branch returns a complete report: SuperGrok Weekly Credits with usedFraction: 0.34, window.durationMs: 604800000, resetsAt matching currentPeriod.end, scope.accountId populated from the token subject, and metadata.email enriched via userinfo. No credential errors, no fanout noise.
  3. Downstream consumer check — I additionally ran the report through the normalizer of a strict quota-rendering consumer (the omp-sub-burndown-indicator plugin, which requires a stable per-account identity + finite usedFraction + window.durationMs + resetsAt): 1 snapshot, 0 diagnostics. So this report shape is sufficient for ecosystem consumers that drop ambiguous/incomplete reports from other providers.

CI: the two red checks look unrelated to this change — Test coding-agent native/unit (TS) fails in eval Julia prelude helpers > supports output ranges… on a 60s timeout (eval/prelude bucket, not packages/ai), and Test TS workspace fast was already re-triggered once as a flake.

This fixes #5065 and I'd love to see it land. If it helps, I'm happy to shepherd: rebase onto current main, address review leftovers, or carry a successor PR with you as co-author — whichever you prefer, @any-victor.

@any-victor

Copy link
Copy Markdown
Contributor Author

hey @bnivanov! thanks for the offer, I can rebase this now, but as I mentioned on the previous comment idk if this will get approved. Though I have opened this #6069 so that we may be able to fully implement this feature in a plugin. I'll also link to that PR the plugin version of this.

@any-victor
any-victor force-pushed the feat/xai-oauth-usage branch from b92246b to 3c83f7e Compare July 20, 2026 19:12
@any-victor

Copy link
Copy Markdown
Contributor Author

@bnivanov rebased!

@bnivanov

bnivanov commented Jul 20, 2026

Copy link
Copy Markdown

Thank you!! Kimi K3 did a good job lol, I am glad we are making progress on this as I am intending to push grok to its limits ha. Appreciate the help @any-victor!

Reject bearer-token redirects so OAuth credentials cannot reach an
unvalidated host.
Keep optional identity enrichment bounded and let the dedicated bearer
fall back past incompatible stored keys.
Rebase onto 16.4.0 landed the entry under the released section; keep
release intent under Unreleased until the next cut.
Document why xai-oauth usage skips generic api_key env fallbacks, pin
billing URLs to https grok.com hosts, and keep weekly windows visible
across period rollover so /usage does not go blank mid-refresh.
Move the changelog entry back under Unreleased after the 16.4.1 cut,
and send X-XAI-Token-Auth: xai-grok-cli on CLI billing so usage polls
match the official Grok CLI host fingerprint.
Prior run failed streamPiNative + codex SSE timeout under parallel
load; all SuperGrok usage tests passed. Empty commit to re-run checks.
Rebase onto 16.4.2 landed the entry under the released section; keep
release intent under Unreleased until the next cut.
Rebase onto current main left the entry under the already-released
16.4.3 section; keep release intent under Unreleased until the next cut.
@any-victor
any-victor force-pushed the feat/xai-oauth-usage branch 2 times, most recently from b6ced5f to 1dd1ec6 Compare July 22, 2026 03:47
@any-victor

Copy link
Copy Markdown
Contributor Author

@roboomp Addressed in the current head. The SuperGrok usage provider is OAuth-only: supports() requires an OAuth credential with an access token, so shared XAI_API_KEY credentials cannot reach cli-chat-proxy.grok.com. The branch also includes the user-visible change under the Unreleased packages/ai changelog section. The focused OAuth usage/auth-storage coverage verifies the credential gate.

@can1357
can1357 merged commit 38061a5 into can1357:main Jul 23, 2026
19 checks passed
musingfox added a commit to musingfox/oh-my-pi that referenced this pull request Jul 23, 2026
can1357#4874 only parsed weekly ?format=credits (creditUsagePercent). Unified
accounts often omit those fields or mark isUnifiedBillingUser while the
default /v1/billing payload still exposes monthlyLimit/used, so omp usage
reported "no usage data". Fall back to and merge the monthly shape.
can1357 pushed a commit that referenced this pull request Jul 23, 2026
#4874 only parsed weekly ?format=credits (creditUsagePercent). Unified
accounts often omit those fields or mark isUnifiedBillingUser while the
default /v1/billing payload still exposes monthlyLimit/used, so omp usage
reported "no usage data". Fall back to and merge the monthly shape.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auth Authentication and login flows feat provider:xai Provider-specific issues for xai providers LLM provider-specific issues review:p2 triaged vouched Passed the vouch gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

xai-oauth / Grok never appears in /usage or omp usage

6 participants