Repository navigation
feat: meter Pinecone query read units by family and caller - #24383
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
There was a problem hiding this comment.
All reported issues were addressed across 7 files
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
|
Required native Luna review is complete at head Round 1 requested integer-only read units, matching the pinned 7.3.0 SDK. Commit Round 2 was a read-only delta review routed by Exact-head CI remains in progress; source review does not establish deployed metric ingestion. Rollout lead owns merge and the next backend train. Local collected review artifact: |
Record Pinecone provider query attempts and returned read units by the shared namespace family, actual route mode (
legacyorpartitioned), and a closed set of explicit callers. Unknown labels map toother; no UID or partition namespace is exported. Missing usage contributes no inferred units and has a separate counter. Metering errors are swallowed and counted.All 11 query sites in
backend/database/vector_db.pypass their function name toOwnerIndex.query. Serving account-purge freshness verification is the one direct_querybypass and is separately metered asaccount_purge_verify, preserving headers, retries and routing. Operator migration queries, fetch/list and mutations are outside this query meter. No route/copy/mutation behavior changes; ships on without a flag.Live response-shape proof: pinned
pinecone==7.3.0,memories-backend-dev, synthetic-owner-only query, no matches or writes, metadata/values disabled. Response typeQueryResponse, usage typepinecone.core.openapi.db_data.model.usage.Usage,usage.read_units == 1(integer). The meter handles SDK attributes and mapping shapes. No production account data read.Integrated local validation on
648ab1813aa5afd12652f2b3083654394ee1d87a: 424 tests passed vianice -n 19 bash backend/test.sh, selected withBACKEND_UNIT_TEST_FILE_LISTand run throughomi-heavy(26 metering, 53 owner-routing, 345 storefront tests). Coverage includes real SDK models, integer/zero/missing/invalid units, fraction/float/boolean rejection, bounded labels, failed provider calls, exporter failures, preserved arguments/results, both account-purge copies, sync/async caller wiring and sync scrape admission. The independent storefront baseline fix is included from main (#24389); the PR diff remains seven meter files.make preflightandscripts/pr-preflight --metadata-only --pr-body-file <draft>passed with the committed candidate and tracked files unchanged throughout. A local no-network smoke drove the actualOwnerIndex.queryandutils.metrics.metrics_responsewith a pinned SDK response: the default-registry exporter contained one query and one read unit, and no owner label. Usage validation follows the pinned client's integer-only data-plane schema. GCP read tier:ro-prod; only infrastructure metric aggregates were read. Luna review and GitHub CI are separate merge gates.Read-only
ro-prodGMP aggregates on 2026-11-02 confirm existing Cloud Run application scraping for backend, desktop-backend, sync, sync-backfill, integration and notifications. The sync sidecar allowlist now admitsomi_pinecone_.*. GKE listen/pusher use authenticated local Prometheus/metricsjobs but were absent from GMP; the GPU batch worker uses a separate registry and performs no Pinecone queries. The local Prometheus sync bridge does not import these counters. Details and acceptance commands:backend/docs/runbooks/pinecone-query-metering.md.Dashboard-free query: which caller spends the most reported read units over 24h, for the GMP Cloud Run scope:
Inspect
omi_pinecone_query_missing_usage_totalandomi_pinecone_query_metering_errors_totalalongside it. GMP totals do not include listen/pusher until that collection gap is resolved or local series are analyzed separately. This meter does not establish a root cause or savings.Deployment: rollout lead to include backend image plus revision-local sync scrape profile in the next backend train; listen/pusher need their own image rolls to emit this instrumentation. Verify running release identity and newly emitted samples after deploy. No deployment performed by this lane.
Product invariants affected
none