Skip to content

Stabilize hosted voice action validation - #72

Merged
alex-w-99 merged 1 commit into
mainfrom
fix/hosted-voice-test-reliability
Sep 14, 2026
Merged

alex-w-99 merged 1 commit into
mainfrom
fix/hosted-voice-test-reliability

Conversation

@alex-w-99

@alex-w-99 alex-w-99 commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Executive Summary

This PR makes the hosted voice action scenario less sensitive to speech turn boundaries and more diagnostic when it fails.

  • Uses one short spoken instruction and primes the call with its post-call purpose.
  • Requires the current marker and SMS intent on both call-leg transcripts.
  • Bumps the plugin version to 0.2.12.

Description

The hosted scenario now gives Voice AI a concise, single-sentence request instead of a multi-sentence instruction that can be split across speech turns. Its initiating message also states that the call should complete a spoken request and record a post-call action.

The persistence gate independently verifies what the driver spoke and what the agent-under-test leg transcribed before accepting the durable open action. Failure output reports bounded predicates and counts without exposing transcript content.

Reason

Natural pauses in the prior prompt could create partial turns and make the hosted-model test nondeterministic. The prior evidence also could not distinguish a spoken request from one actually heard by Voice AI.

Decisions

  • Scope: Hardened the live test and its evidence gates without changing plugin runtime behavior because the observed failure occurred before post-call plugin execution.
  • Diagnostics: Kept failure diagnostics content-free so CI output identifies the failed gate without publishing call content.

Testing

  • uv run --with pytest pytest -q: 567 passed, 23 skipped.
  • Hosted outbound voice action scenario: CI should hear the full marker-bearing request on both call legs, persist one matching open action, and send exactly one post-call SMS. Not run locally because it requires live credentials and telephony infrastructure.
  • git diff --check: passed.

@alex-w-99
alex-w-99 marked this pull request as ready for review September 14, 2026 02:34
@alex-w-99
alex-w-99 merged commit 1f9ec22 into main Sep 14, 2026
15 checks passed
@alex-w-99
alex-w-99 deleted the fix/hosted-voice-test-reliability branch September 14, 2026 02:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant