Skip to content

feat(usage-ingest): accept authenticated usage events - #7229

Merged
johnnyeric merged 1 commit into
johnnyeric/bounded-stream-readerfrom
johnnyeric/usage-publisher-endpoint
Oct 8, 2026
Merged

johnnyeric merged 1 commit into
johnnyeric/bounded-stream-readerfrom
johnnyeric/usage-publisher-endpoint

Conversation

@johnnyeric

@johnnyeric johnnyeric commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Add authenticated POST /usage to validate the existing usage event and send it to USAGE_INGEST_QUEUE, preserving the supplied ID. Return 202 only after queue acceptance. Authenticate before body access, enforce strict UTF-8 and actual input/serialized event limits of 120,000 bytes, strip unknown keys, and return safe failures for invalid requests or enqueue errors.

Extract the current request/response/Bouncer schemas and usage enums into @kilocode/usage-contracts, whose only runtime dependency is Zod. The Worker imports its real package export with no web-shared dependency or resolver aliases. Existing DB/web-shared exports remain compatible, and exact DB/type alignment assertions remain in web-shared. Preserve the default-null reasoning_setting rollout for older senders; the queue fixture includes that field so exact forwarding and byte-boundary checks use the parsed event shape.

Verification

  • Serialization overflow is handled as 413 before enqueueing. The new deeply nested Bouncer regression fails with a real RangeError before the fix and passes after it. All 16 Worker tests, Worker typechecking, lint and formatting pass. A fresh workerd HTTP/Queue check returns 413 for a schema-valid 21 KB nested payload with no enqueue; the normal control returns 202 and exactly one local Queue receipt.
  • 90 focused tests pass on this intermediate layer: 24 current schema tests moved to Vitest, 16 publisher tests including startup and localhost HTTP through the actual configured workerd entry, and 50 existing usage-client/diagnostics tests with database setup disabled.
  • 13 real localhost HTTP probes matched expected responses; exactly three accepted events reached the local Wrangler/Miniflare Queue and temporary consumer with all validated fields and IDs preserved. Covers an older sender omitting reasoning_setting, Unicode/Bouncer data, unknown-key stripping and the 120,000-byte boundary; rejected requests produced no extra receipts. All temporary servers stopped.
  • Contract, DB, web-shared, AI gateway, web (including its tRPC build) and Worker typechecks, focused lint/formatting and frozen ignore-scripts install pass after rebase. Actual workerd startup and Queue delivery verify the configured Worker; previous production/staging bundle isolation evidence remains unchanged by the import cleanup.
  • All current-main contract declarations and 24 schema assertions are preserved through extraction. Reader patch is unchanged after rebase; worker-utils has 788 passing tests and session-ingest has 1,226 passing tests with three existing skips. Both package typechecks pass. The Worker and canonical package sources are unchanged by the upper import-cleanup layer.

Visual Changes

N/A

Reviewer Notes

Depends on #7228 and targets johnnyeric/bounded-stream-reader: one commit, 22 files, +675/-190. Merge the prerequisite first, then retarget this PR to main. Direct-import cleanup is the separate dependent draft #7258.

202 confirms queue acceptance; a future consumer performs processing. No consumer, gateway activation or existing Vercel API integration is included. Public Worker/preview URLs remain disabled. Verification used only local synthetic inputs; no remote queue, deployment, live PostgreSQL or billing operations were performed. The usage-client unit suite mocks fetch and does not replace database-backed writer tests.

@johnnyeric
johnnyeric added this pull request to stack #7231 October 6, 2026 21:30
@johnnyeric
johnnyeric force-pushed the johnnyeric/usage-publisher-endpoint branch 2 times, most recently from 0fe3702 to d447a1c Compare October 6, 2026 23:06
@johnnyeric
johnnyeric marked this pull request as ready for review October 6, 2026 23:07
@kilo-code-bot

kilo-code-bot Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

The previously reported pre-enqueue JSON.stringify gap is fixed at HEAD (the serialization/size check now has its own try/catch returning the generic 413 while queue failures keep 503), and the authenticated POST /usage publisher, auth-before-body ordering, bounded byte reads, size enforcement, awaited queue acceptance, and the Zod-only @kilocode/usage-contracts extraction with compatibility re-exports are otherwise sound. No new issues found.

Files Reviewed (22 files)
  • packages/db/package.json
  • packages/db/src/schema-types.ts
  • packages/usage-contracts/README.md
  • packages/usage-contracts/package.json
  • packages/usage-contracts/src/index.test.ts
  • packages/usage-contracts/src/index.ts
  • packages/usage-contracts/src/schema-types.ts
  • packages/usage-contracts/tsconfig.json
  • packages/usage-contracts/vitest.config.ts
  • packages/web-shared/AGENTS.md
  • packages/web-shared/package.json
  • packages/web-shared/src/lib/ai-gateway/usage-record-contract.ts
  • pnpm-lock.yaml
  • services/usage-ingest/README.md
  • services/usage-ingest/package.json
  • services/usage-ingest/src/index.runtime.test.ts
  • services/usage-ingest/src/index.test.ts
  • services/usage-ingest/src/index.ts
  • services/usage-ingest/src/publish-usage.ts
  • services/usage-ingest/tsconfig.json
  • services/usage-ingest/vitest.config.ts
  • services/usage-ingest/wrangler.jsonc
Previous Review Summaries (4 snapshots, latest commit c31e540)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit c31e540)

Status: 1 Issue Found | Recommendation: Address before merge

Executive Summary

The authenticated POST /usage publisher, the @kilocode/usage-contracts extraction, and the compatibility re-exports are otherwise sound, but the pre-enqueue JSON.stringify size check sits outside the parse try/catch, so a schema-valid pathological bouncer_usage_event.payload can surface as an unhandled 500 instead of a 413.

Overview

Severity Count
CRITICAL 0
WARNING 0
SUGGESTION 1
Issue Details (click to expand)

SUGGESTION

File Line Issue
services/usage-ingest/src/publish-usage.ts 49 Pre-enqueue JSON.stringify runs outside the parse try/catch; a schema-valid deeply nested bouncer_usage_event.payload can throw and surface as an uncontrolled 500
Files Reviewed (22 files)
  • packages/db/package.json
  • packages/db/src/schema-types.ts
  • packages/usage-contracts/README.md
  • packages/usage-contracts/package.json
  • packages/usage-contracts/src/index.test.ts
  • packages/usage-contracts/src/index.ts
  • packages/usage-contracts/src/schema-types.ts
  • packages/usage-contracts/tsconfig.json
  • packages/usage-contracts/vitest.config.ts
  • packages/web-shared/AGENTS.md
  • packages/web-shared/package.json
  • packages/web-shared/src/lib/ai-gateway/usage-record-contract.ts
  • pnpm-lock.yaml
  • services/usage-ingest/README.md
  • services/usage-ingest/package.json
  • services/usage-ingest/src/index.runtime.test.ts
  • services/usage-ingest/src/index.test.ts
  • services/usage-ingest/src/index.ts
  • services/usage-ingest/src/publish-usage.ts - 1 issue
  • services/usage-ingest/tsconfig.json
  • services/usage-ingest/vitest.config.ts
  • services/usage-ingest/wrangler.jsonc

Fix these issues in Kilo Cloud

Previous review (commit 955f9ef)

Status: 1 Issue Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 0
SUGGESTION 1
Issue Details (click to expand)

SUGGESTION

File Line Issue
services/usage-ingest/src/publish-usage.ts 49 Pre-enqueue JSON.stringify runs outside the parse try/catch; a schema-valid deeply nested bouncer_usage_event.payload can throw and surface as an uncontrolled 500
Files Reviewed (22 files)
  • packages/db/package.json
  • packages/db/src/schema-types.ts
  • packages/usage-contracts/README.md
  • packages/usage-contracts/package.json
  • packages/usage-contracts/src/index.test.ts
  • packages/usage-contracts/src/index.ts
  • packages/usage-contracts/src/schema-types.ts
  • packages/usage-contracts/tsconfig.json
  • packages/usage-contracts/vitest.config.ts
  • packages/web-shared/AGENTS.md
  • packages/web-shared/package.json
  • packages/web-shared/src/lib/ai-gateway/usage-record-contract.ts
  • pnpm-lock.yaml
  • services/usage-ingest/README.md
  • services/usage-ingest/package.json
  • services/usage-ingest/src/index.runtime.test.ts
  • services/usage-ingest/src/index.test.ts
  • services/usage-ingest/src/index.ts
  • services/usage-ingest/src/publish-usage.ts - 1 issue
  • services/usage-ingest/tsconfig.json
  • services/usage-ingest/vitest.config.ts
  • services/usage-ingest/wrangler.jsonc

Executive Summary

One low-severity hardening item: the pre-enqueue reserialization is not guarded by the parse try/catch, so a schema-valid but pathological payload could surface a 500 instead of a 4xx; auth ordering, bounded reads, size limits, and the contract extraction are otherwise sound.

Previous review (commit 6a31f14)

Status: No Issues Found | Recommendation: Merge

Executive Summary

The new authenticated POST /usage publisher, the shared @kilocode/usage-contracts extraction, and the compatibility re-exports were checked for auth ordering, bounded reads, size enforcement, queue awaiting, and type alignment; no new issues were found.

Files Reviewed (22 files)
  • packages/db/package.json
  • packages/db/src/schema-types.ts
  • packages/usage-contracts/README.md
  • packages/usage-contracts/package.json
  • packages/usage-contracts/src/index.test.ts
  • packages/usage-contracts/src/index.ts
  • packages/usage-contracts/src/schema-types.ts
  • packages/usage-contracts/tsconfig.json
  • packages/usage-contracts/vitest.config.ts
  • packages/web-shared/AGENTS.md
  • packages/web-shared/package.json
  • packages/web-shared/src/lib/ai-gateway/usage-record-contract.ts
  • pnpm-lock.yaml
  • services/usage-ingest/README.md
  • services/usage-ingest/package.json
  • services/usage-ingest/src/index.runtime.test.ts
  • services/usage-ingest/src/index.test.ts
  • services/usage-ingest/src/index.ts
  • services/usage-ingest/src/publish-usage.ts
  • services/usage-ingest/tsconfig.json
  • services/usage-ingest/vitest.config.ts
  • services/usage-ingest/wrangler.jsonc

Previous review (commit d447a1c)

Status: No Issues Found | Recommendation: Merge

Files Reviewed (11 files)
  • packages/web-shared/AGENTS.md
  • pnpm-lock.yaml
  • services/usage-ingest/README.md
  • services/usage-ingest/package.json
  • services/usage-ingest/src/index.runtime.test.ts
  • services/usage-ingest/src/index.test.ts
  • services/usage-ingest/src/index.ts
  • services/usage-ingest/src/publish-usage.ts
  • services/usage-ingest/tsconfig.json
  • services/usage-ingest/vitest.config.ts
  • services/usage-ingest/wrangler.jsonc

Reviewed by deepseek-v4.1-flash · Input: 72.2K · Output: 14.1K · Cached: 912.1K

Review guidance: REVIEW.md from base branch johnnyeric/bounded-stream-reader

Comment thread packages/web-shared/AGENTS.md Resolved
@johnnyeric
johnnyeric force-pushed the johnnyeric/usage-publisher-endpoint branch from d447a1c to 6a31f14 Compare October 7, 2026 15:16
@johnnyeric
johnnyeric force-pushed the johnnyeric/usage-publisher-endpoint branch 2 times, most recently from 955f9ef to c31e540 Compare October 7, 2026 17:21
Validate bounded usage events and await queue acceptance. Extract the shared
Zod-only usage contracts while preserving compatibility exports and DB checks.
@johnnyeric
johnnyeric force-pushed the johnnyeric/usage-publisher-endpoint branch from c31e540 to 9b6e920 Compare October 7, 2026 17:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants