Skip to content

Conformance check: DTMF logged-as-sent vs present-in-delivered-audio #15

Description

@quantumCF

The failure class

A tool log records DTMF digits as sent while the far-end IVR never hears
them. Public evidence the class exists in the wild: a provider support
thread
(support thread 22881851)
documents DTMF digits logged as sent while the far-end IVR does not detect
the tones. The class is a log-versus-audio
disagreement, and hotato's stance on that is already fixed: evidence over
logs. The tool_call assertion counts only ingested trace spans because an
agent's words claiming a tool ran do not count. This check applies the same
move to DTMF: a "sent" claim is conformant only if the tones are audibly
present in the delivered audio at the claimed time.

What exists to build on

  • The scripted caller sends DTMF as a first-class action:
    dtmf={action,digits} nodes in src/hotato/caller.py, digits validated
    against ^[0-9A-Da-d*#]{1,64}$, with dtmf as a recorded evidence kind.
    Those evidence events supply the claimed digits and the send window when
    the check runs against a captured call.
  • The failure-record vocabulary (src/hotato/failure_record.py) already
    carries a dtmf condition with the public reason "The declared DTMF
    condition failed.", so a failing check has a home in the existing artifact
    shape.
  • corpus/classes/build_classes.py is the deterministic render pattern, and
    DTMF is the easiest audio in the codebase to render exactly: each digit
    is a pure sine pair from the standard 4x4 row/column frequency table,
    reproducible with stdlib math.

The check and the fixture

  1. Fixture. A deterministic two-channel WAV whose caller channel carries
    rendered tone pairs for a stated digit string at stated offsets and
    durations, plus a defect render where one digit's tones are absent and
    the claim is unchanged. The label states digits, offsets, durations, and
    amplitudes.
  2. Check. One function: given a WAV, a channel, a digit string, and a
    claimed send window, assert each digit's two frequencies are present
    in-window (Goertzel or FFT bin energy against a stated threshold, about
    thirty lines of stdlib) and report pass, fail, or inconclusive per digit,
    with the measured energies as evidence.

Functional scope, stated plainly: the check measures tone presence in the
recording it is given. What the far end did with the tones stays with the
far end; delivered-audio presence is the strongest claim a recording
supports, and it is exactly the claim the cited thread needed.

Done looks like

  • The fixture regenerates byte-identically (--check in the builder
    pattern).
  • One test: the pass render passes every digit, the defect render fails on
    the named digit and only that digit.
  • The check function has a docstring stating the threshold, the window
    semantics, and the per-digit result shape.
  • python -m pytest -q and python scripts/copy_lint.py both pass.

Size

One fixture, one detector function, one test. Good first issue: the tone
table is public, the render pattern exists, and the check is standalone with
no engine changes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions