Skip to content

[EPIC] Data Accuracy & Eval Readiness #205

Description

@gjanco

Outcome

Metrics used to evaluate AURA are defined consistently and match the underlying provider and runtime behavior.

Scope

  • Token and usage semantics.
  • End-to-end and phase timing.
  • SSE phase boundaries and summaries.
  • Measurement needed to attribute benchmark failures and performance differences.

Anonymous adoption telemetry and production-install classification belong to the Adoption Measurement initiative.

Complete when

  • Token counts are validated against provider ground truth.
  • Timing accounts for the full measured interval without double-counting.
  • Events use consistent semantics across single-agent and orchestration paths.
  • Benchmark reports can distinguish model, AURA, tool, and harness time where the data permits.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions