Skip to content

Label Analytics alert jobs for reliable cost attribution - #9564

Open
3mdistal wants to merge 3 commits into
mainfrom
alice/analytics-alert-job-labels-1ed067ed
Open

3mdistal wants to merge 3 commits into
mainfrom
alice/analytics-alert-job-labels-1ed067ed

Conversation

@3mdistal

Copy link
Copy Markdown
Contributor

Problem

Analytics alert batches use the same SQL wrapper as dashboards. Cost reporting cannot reliably distinguish their intent from query text alone, so a SQL-shape classifier can move genuine dashboard costs into the alert category.

Approach

Record the workload when Analytics submits the BigQuery job. This adds configuration.labels.agent_native_workload=alert only when the existing server-owned usage context identifies an alert. Dashboard and unattributed submissions receive no new label, even if their SQL contains an alert marker. The label carries no user, organization, dashboard or panel identifiers.

Safety and operations

The change preserves job IDs, SQL, query results, credentials, cache/resume behavior and billed-byte caps. Labels describe the submitting app's context; they are not an independent attestation against other holders of the same warehouse credentials.

This PR records provenance for newly created jobs. A separate warehouse-classifier patch is prepared locally and is not included or published here. Existing jobs are not relabelled, October attribution remains unresolved, and no cost reduction or measured savings is claimed. No warehouse query, historical backfill or production promotion ran during this work.

Verification

  • 210 targeted tests passed: the BigQuery runner, usage accounting, alert batching and alert byte caps.
  • The fake-provider regression enters through the real batch evaluator and backend, then inspects the outgoing job label and existing cap. Removing the evaluator's attribution context makes it fail specifically at the label assertion.
  • Concurrent runner cases verify alert/dashboard context isolation, unchanged query payloads and absence of private identifiers in labels. Removing the submission label makes the regression fail.
  • Analytics typecheck, changed-file formatting, compiler-strictness, silent-coercion and oracle guards passed on the committed source. Named-file lint has no errors and one warning on an unchanged schema-conversion line already present in the base.
  • Independent Sol review found no submission/classification defect; the requested producer-entry-point coverage is included. Source hashes matched the isolated test machine. No rendered UI changed; native BigQuery acceptance and live label ingestion remain unverified.

Review focus

The label must come only from server-owned alert context at submission. Confirm the producer-to-provider path and concurrent negative controls preserve that boundary without altering query execution.

@builder-io-integration builder-io-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Builder reviewed your changes — looks good ✅

Review Details

Code Review Summary

PR #9564 adds a BigQuery job label (agent_native_workload=alert) to submissions only when the server-side usage context identifies the query as an Analytics alert. The change addresses the attribution gap without changing dashboard or unattributed job payloads; the accompanying specs exercise both the isolated submission path and the real alert-batch evaluator. The implementation is narrowly scoped and places the label in the BigQuery job configuration where expected.

Risk assessment: Standard. Two independent reviews found no confirmed bugs, security issues, or performance regressions. The server-owned attribution signal and tests covering concurrent alert/dashboard/unattributed contexts are good safeguards against accidentally labeling dashboard costs. No actionable findings.

🧪 Browser testing: Skipped — PR only modifies backend code and tests, with no UI impact.

@github-actions

Copy link
Copy Markdown
Contributor

Visual recap — generation failed

Recap authentication failed — PLAN_RECAP_TOKEN was rejected; it may be expired, revoked, or minted for another Plans app. Personal token: run npx @agent-native/core@latest connect <app-url> --client codex, then npx @agent-native/recap-cli@latest recap setup. Org service token: an owner/admin can mint a 10-year replacement with npx -y @agent-native/core@latest connect <app-url> --service-token <name> --ttl-days 3650. Update this repo's secret.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants