Add HD realtime voice and repair live CI connection setup

[dimavrem22]

Executive Summary

Add HD realtime voice and repair connection setup failures in live checks.

• Negotiate 16 kHz PCM16 call audio and stream-convert it to and from 24 kHz realtime audio.
• Preserve legacy call audio and retry only failed HTTP connection establishment in live tests.

Description

The realtime bridge advertises PCM16 in the call WebSocket handshake, reads the negotiated start descriptor, and keeps independent streaming conversion state for each direction. Partial samples survive frame boundaries; output state resets on interruption and audio completion. Unsupported formats fail explicitly instead of producing incorrectly encoded speech.

Live tests use bounded connection-establishment retries without retrying potentially delivered requests. The hosted-call peer makes its request once, without reminders or corrective coaching; exact action, recipient, marker, and single-send assertions remain unchanged. Python 3.13 joins the unit matrix to exercise the conditional audio conversion dependency.

Reason

Realtime voice should retain wideband call audio instead of forcing it through 8 kHz μ-law. Post-call reconciliation now explicitly preserves exact caller-specified SMS wording without replaying an accepted send. Recent live failures also exposed connection setup errors that can be retried safely without duplicating user actions.

Decisions

• Audio contract: Request 16 kHz call PCM and resample to the realtime API's required 24 kHz format; setting a sample-rate label alone would distort playback.
• Compatibility: Preserve 8 kHz μ-law when an older call stream advertises it or omits its descriptor.
• Hosted-call proof: One caller request only. The agent must persist the action, wake after hangup, and send the exact SMS autonomously; no test reminders or corrective coaching.
• Retries: Retry only connection establishment, keeping action, delivery, and post-call assertions unchanged.

Testing

• Claude Code realtime calls: verify two-way speech, correct playback speed, interruption, transcript persistence, normal post-call handling, and exact-call 16 kHz PCM negotiation.
• Offline suite: 560 passed; live checks correctly skipped without credentials.
• Host contract suite: 5 passed.
• Audio tests cover actual PCM bytes, tone frequency/gain/duration, split-sample chunks, silence, legacy encoding, negotiated wire format, and interruption resets.
• Unassisted full-stack CI passed on c0bf236: all 13 jobs and all 21 PR checks passed. The original single-request hosted scenario passed in 87 seconds, proving durable action registration before hangup, autonomous agent wakeup and confirmed SMS execution, completed post-call receipt, and exactly one current-marker SMS to the caller after the duplicate grace window. Realtime also verified negotiated 16 kHz PCM and two-way speech. Earlier reminder-assisted results are superseded.

[greptile-apps]

Greptile Summary

Adds negotiated 16 kHz PCM call audio with stateful bidirectional conversion and narrows live-test retries to connection establishment.

• Advertises and validates the HD call-media format while retaining legacy 8 kHz μ-law support.
• Adds streaming resampling, frame-boundary handling, and interruption/completion resets.
• Expands audio, gateway, hosted-call, transport, and Python 3.13 test coverage.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
inkbox_claude/audio.py	Introduces stateful mono PCM16 resampling and explicit negotiation for HD PCM and legacy μ-law call streams.
inkbox_claude/realtime.py	Converts call audio independently of the realtime API’s 24 kHz PCM session and resets outbound conversion state at playback boundaries.
inkbox_claude/gateway.py	Requests 16 kHz signed little-endian PCM for realtime calls and strengthens exact post-call SMS wording guidance.
tests/live/conftest.py	Enables bounded HTTP connection-establishment retries for credentialed live-test sessions.
pyproject.toml	Adds the conditional audioop compatibility dependency required on Python 3.13 and later.

Sequence Diagram

sequenceDiagram
    participant Caller
    participant Inkbox as Inkbox Call WS
    participant Bridge as Realtime Bridge
    participant OpenAI as Realtime API
    Caller->>Inkbox: Call audio
    Inkbox->>Bridge: start(media_format)
    Bridge->>Bridge: Configure per-direction converters
    Inkbox->>Bridge: 16 kHz PCM or legacy 8 kHz PCMU
    Bridge->>OpenAI: Resampled 24 kHz PCM
    OpenAI->>Bridge: 24 kHz PCM response audio
    Bridge->>Inkbox: Resampled negotiated call audio
    Bridge->>Inkbox: audio_done / clear

Loading

_{Reviews (4): Last reviewed commit: "test: restore unassisted hosted voice re..." | Re-trigger Greptile}