Skip to content

Add audio pronunciation (TTS) for words and hints #4

Description

@llinsss

Problem

The entire hint and story system (agent/hint_generator.py, agent/story_mode.py) is text-only. The target users are children learning to read — many can't decode the hint text itself, which undermines the product's core purpose.

Scope

  • Integrate a TTS provider (e.g. Amazon Polly, since the project already uses AWS/boto3 for Bedrock) to generate audio for: the target word, each hint level, and generated stories.
  • Add an endpoint (or extend existing ones) to return an audio URL/stream alongside text, e.g. POST /api/v1/hint response gains an audio_url field.
  • Cache generated audio (don't regenerate on every request for the same word/hint) — a local file cache keyed by content hash is sufficient.
  • Fall back gracefully (text-only) if the TTS provider is unavailable or unconfigured, mirroring the existing Bedrock fallback pattern.

Acceptance criteria

  • /api/v1/hint and /api/v1/story responses include an audio reference when TTS is configured.
  • Requesting the same word/hint twice reuses cached audio (verified by mocking the TTS client and asserting it's called once).
  • With TTS unconfigured/erroring, endpoints still return valid text-only responses (test mirrors the existing test_bedrock_story_fallback_on_error pattern).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    GrantFox OSSIssue tracked in GrantFox OSSMaybe RewardedIssue may be eligible for a GrantFox rewardOfficial Campaign | FWC26Campaign: Official Campaign | FWC26accessibilityAccessibility improvementsenhancementNew feature or requestttsText-to-speech/audio

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions