Skip to content

RFC: a share-safe role-play fixture format (scripted defects, recorded on purpose) #16

Description

@quantumCF

Why an RFC

The corpus accepts role-played recordings today: source_type: "role-played" is a first-class value in corpus/label.schema.json, and
CONTRIBUTING.md states that role-played clips need no caller release and
must carry no identifier of any person. Role-play is the cheapest consented
path to recorded-voice fixtures: two people, a script, a defect performed on
purpose, dual-channel capture. What this RFC standardizes is the recipe, so
contributors can produce comparable clips repeatably and reviewers can
accept them quickly.

What exists to build on

  • corpus/label.schema.json: the contribution shape with provenance,
    consent, PII, and attestation fields; corpus/validate.py checks a
    (recording, label) pair locally until it prints PASS.
  • docs/CORPUS-GOVERNANCE.md: the governing consent template, PII strip
    list, same-duration redaction rule, and the no-PHI line.
  • docs/SUBMITTING.md and the corpus_submission.yml issue form: the
    submission path a role-play contribution rides today.
  • The synthetic suites (corpus/suites/) as the deterministic floor that
    role-played clips sit above: rendered timings prove the scorer matches its
    spec; recorded voices prove it measures a live line.

Proposed, for comment

  1. A role-play script format. One short file per failure class (markdown
    or JSON, open question) carrying: the speaker turns, the deliberate
    defect, and the timing beats to hit ("agent begins the readback; caller
    cuts in about two words in"). Two people plus any stack can then record
    the same scenario on different infrastructure, which makes the clips
    comparable across contributions.
  2. Where scripts live. Proposal: corpus/role-play/, one script per
    file, indexed from corpus/README.md.
  3. The minimum label. source_type: "role-played"; both role-players
    named in the attestation as the audible, consenting parties; a
    hand-labeled caller_onset_sec; and no third-party identifiers by
    construction, because scripts contain placeholder data only (555 numbers,
    example.com addresses, invented names).
  4. A one-line release for role-players. The governance doc's release
    paragraph, reduced to the role-play case: the performers are willing
    participants recording for the corpus, so the release is a line each,
    attached to the label. Open question: does that line live in the script
    file, the label, or both.
  5. Tiering. Whether role-played clips enter the same suites as
    synthetic scenarios or a separate tier with their own manifest, given
    they carry human timing variance by design.

What a good first comment looks like

Pick one failure class from the seeded fixture issues (the sample-boundary
interruption, the structured-utterance pause, the tool-race readback) and
draft its script in your preferred format. A concrete script settles format
questions faster than abstract ones.

Done looks like

The thread converges on a documented format: a corpus/role-play/ section
(or the agreed home) with at least one script, a role-play section in
docs/SUBMITTING.md, and the first accepted role-play contribution
following it end to end.

Metadata

Metadata

Assignees

No one assigned

    Labels

    help wantedExtra attention is neededrfcRequest for comments: a proposal open for discussion

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions