AI bilingual subtitles that stream in live · click any word for an instant dictionary · listening drills built in.
Designed for Chinese learners. Built like Apple would.
Demo · On your phone · Quick Start · 中文文档
Most language apps make you study their content. Lingua Nova flips that: paste a link to any YouTube video you actually want to watch, and it becomes a complete lesson — synchronized bilingual subtitles, vocabulary highlighted at your level, a tap-to-define dictionary, and listening drills. The friction of looking things up disappears; only the input remains.
| ⚡ Instant | 🎯 Personal | 🔁 Complete |
| English subtitles appear the moment they're fetched — translations stream in live while you watch. Stop anytime; it resumes where it left off. | Set your level on a single axis — Liftoff to Supernova — and the AI highlights only words above it. Reopen any video and continue from your last position. | Watch → look up → collect → drill → export to Anki. The whole learning loop, not just a player. |
The active sentence glides with playback like Apple Music lyrics — buttery even on a 90-minute podcast. Click any sentence to jump the video there. AI-picked vocabulary carries its Chinese gloss as ruby text above the word — visible at a glance, ignorable at speed.
One click on any word in the transcript opens a dictionary card: IPA, part of speech, a context-aware Chinese gloss, an English definition, and an example sentence — with one-tap pronunciation and one-tap save to your vocabulary book. Definitions are cached on disk, so each word costs an API call exactly once, ever. Or flip on dictation mode to blur the English and check yourself by ear.
 Tap-to-define dictionary — IPA · 释义 · TTS · 生词本 |
 Dictation mode — listen first, reveal to check yourself |
The interface is fully responsive — the same synced bilingual transcript, tap-to-define dictionary, and listening tools, sized for a phone. A companion Expo React Native client also ships in mobile/.
Prereqs: Node 18+, Python 3.10+, and a free Gemini API key.
git clone https://github.com/huthvincent/yt-bilingual-app.git
cd yt-bilingual-app
./setup.sh # installs frontend + backend deps
cp .env.example .env # then paste your GEMINI_API_KEY
./start_app.command # macOS one-click (or run the two commands below)Manual start
# terminal 1 — backend
cd backend && source venv/bin/activate && uvicorn main:app --port 8000
# terminal 2 — frontend
cd frontend && npm run devOpen http://localhost:5173, paste a YouTube link, set your level on the axis, and press Start Learning.
| Variable | Where | Purpose |
|---|---|---|
GEMINI_API_KEY |
.env |
Required. Translation, summaries, dictionary. |
WHISPER_MODEL |
.env |
Whisper size for the no-captions ASR fallback (base default). |
CORS_ORIGINS |
.env |
Comma-separated allowed origins — set before deploying publicly. |
API_AUTH_KEY |
.env |
When set, every API call must send X-API-Key. Protects your Gemini quota on public deployments. |
VITE_API_BASE / VITE_API_KEY |
frontend env | Point the web app at a remote backend / send its API key. |
HISTORY_DIR / SUBTITLES_DIR |
env | Relocate the JSON cache / local-show subtitles. |
- Progressive SSE pipeline with resume & self-healing translations
- Tap-to-define dictionary with IPA, TTS, and vocabulary book
- Dictation mode, sentence loop, playback speed, keyboard control
- Vocabulary highlighting on a Liftoff → Supernova level axis
- Anki / CSV export
- Whisper ASR fallback for caption-less videos
- AI pronunciation assessment (speak the sentence, get scored)
- In-app spaced-repetition review
- Chrome extension — start a lesson from any YouTube page
- Mobile parity with the streaming pipeline
Issues and PRs welcome. Fork → git checkout -b feature/amazing → commit → PR.
Architecture, design language, and per-feature specs (spec-kit style) live in docs/ and specs/, with project principles in CONSTITUTION.md. These docs are in Chinese.
MIT — see LICENSE.
cd scripts && APP_URL=http://localhost:5173 node capture.mjs all.