Skip to content

[P1.5-S6-ARI-45] UC3 round 3 acceptance test + report addendum #79

Description

@bjridicodes

Ticket: P1.5-S6-07

Type: Execution | Est: 1.5d

Goal: AC-01 through AC-06 validated on all 10 UC3 incidents with full log retrieval — closes the round 2 UC3 gap and completes Phase 1.5.

Pre-conditions: P1.5-S5-03 (UC3 incidents in ServiceNow), P1.5-S6-06 (UC3 smoke test passing)

Scope:

  • Update tests/acceptance/ground_truth.json: set logs_available: true for all 10 UC3 entries
  • Run all 10 UC3 incidents through ARIA pipeline in tool mode and API mode
  • Score AC-01 through AC-06 (AC-03 now fully evaluated — no N/A for UC3)
  • Append results to tests/acceptance/report_round2.md as "UC3 Round 3 Addendum" section
  • Update p1_5_roadmap.md Phase 1.5 completion criteria table — mark all 6 sprint criteria as complete

Acceptance criteria:

  • AC-03 (log recall) ≥ 80% on UC3 incidents
  • AC-04 (classification accuracy) ≥ 70% on UC3 incidents
  • AC-05 (confidence score) 100% of UC3 notifications include confidence band
  • Report addendum written, reviewed, and committed to main
  • Phase 1.5 completion criteria in p1_5_roadmap.md marked all-green

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1.5-S6S6: GCP Native Service Connectorsphase-1.5All Phase 1.5 issuestaskConfig, data, validation, execution, or docs work

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions