Add langdetect dependency and enhance transcription logic #157
Conversation
- Updated transcription logic in `transcribe.py` to detect language and translate if necessary, improving multi-language support. - Adjusted prompt formatting for better clarity in transcription requests. - Added `langdetect` version 1.0.9 to dependencies in `pyproject.toml`, `requirements-dev.lock`, and `requirements.lock`.
WalkthroughThis change introduces language detection and conditional translation to the audio transcription pipeline. It updates prompt construction for transcription, disables prior chunk context, and adds new translation prompt templates in multiple languages. The Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant TranscribeAPI
participant WhisperModel
participant LangDetect
participant LiteLLM
participant PromptTemplates
User->>TranscribeAPI: Submit audio file, target language
TranscribeAPI->>WhisperModel: Transcribe audio
WhisperModel-->>TranscribeAPI: Transcription text
TranscribeAPI->>LangDetect: Detect language of transcription
LangDetect-->>TranscribeAPI: Detected language
alt Detected language ≠ requested language and requested ≠ "multi"
TranscribeAPI->>PromptTemplates: Render translation prompt
TranscribeAPI->>LiteLLM: Request translation completion
LiteLLM-->>TranscribeAPI: Translated text
TranscribeAPI-->>User: Return translated text
else
TranscribeAPI-->>User: Return transcription text
end
Possibly related PRs
Suggested reviewers
LGTM. Note ⚡️ AI Code Reviews for VS Code, Cursor, WindsurfCodeRabbit now has a plugin for VS Code, Cursor and Windsurf. This brings AI code reviews directly in the code editor. Each commit is reviewed immediately, finding bugs before the PR is raised. Seamless context handoff to your AI code agent ensures that you can easily incorporate review feedback. Note ⚡️ Faster reviews with cachingCodeRabbit now supports caching for code and dependencies, helping speed up reviews. This means quicker feedback, reduced wait times, and a smoother review experience overall. Cached data is encrypted and stored securely. This feature will be automatically enabled for all accounts on May 16th. To opt out, configure ✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
⛔ Files ignored due to path filters (2)
echo/server/requirements-dev.lockis excluded by!**/*.lockecho/server/requirements.lockis excluded by!**/*.lock
📒 Files selected for processing (7)
echo/server/dembrane/transcribe.py(4 hunks)echo/server/prompt_templates/translate_transcription.de.jinja(1 hunks)echo/server/prompt_templates/translate_transcription.en.jinja(1 hunks)echo/server/prompt_templates/translate_transcription.es.jinja(1 hunks)echo/server/prompt_templates/translate_transcription.fr.jinja(1 hunks)echo/server/prompt_templates/translate_transcription.nl.jinja(1 hunks)echo/server/pyproject.toml(1 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (1)
echo/server/dembrane/transcribe.py (1)
echo/server/dembrane/prompts.py (1)
render_prompt(55-88)
⏰ Context from checks skipped due to timeout of 90000ms (2)
- GitHub Check: ci-check-server
- GitHub Check: ci-build-servers (dbr-echo-server, ./echo/server, Dockerfile, dbr-echo-server)
🔇 Additional comments (6)
echo/server/pyproject.toml (1)
89-89: Solid addition of the langdetect dependency! 🚀Adding langdetect at version 1.0.9 is exactly what we need to implement language detection capabilities in the transcription pipeline. This choice demonstrates production-ready engineering.
echo/server/prompt_templates/translate_transcription.fr.jinja (1)
1-5: Elegant French prompt template implementation! ✨This template is clean, efficient, and provides clear instructions for the translation model. The focus on preserving meaning, tone, and context while avoiding modifications demonstrates a deep understanding of translation quality requirements.
echo/server/prompt_templates/translate_transcription.de.jinja (1)
1-5: Excellent German prompt template construction! 💯The German translation template follows the same robust pattern as the other language templates, maintaining consistency across the system. This modular design allows for seamless scaling to support additional languages in the future.
echo/server/prompt_templates/translate_transcription.nl.jinja (1)
1-5: Dutch prompt template perfectly crafted! 🔥The Dutch translation template completes the set of language templates with the same high-quality approach. The consistent structure across all templates will ensure uniform translation behavior regardless of language pairs. Nice work!
echo/server/prompt_templates/translate_transcription.en.jinja (1)
1-5: Prompt reads well, ship itEverything’s crisp; no action items.
echo/server/dembrane/transcribe.py (1)
83-87: Good call on explicit default promptsExplicit language declarations reduce hallucinated code-switching – nicely done.
- Changed variable name from `response` to `llm_translation_response` for clarity in the transcription process.
There was a problem hiding this comment.
Actionable comments posted: 1
🔭 Outside diff range comments (1)
echo/server/dembrane/transcribe.py (1)
35-79: 🧹 Nitpick (assertive)Consider adding metrics/logging for translation performance
Your transcription + translation pipeline is solid, but monitoring will be crucial as this rolls out. Consider adding:
- Performance metrics to track the time spent in transcription vs. translation
- Success rates for language detection
- Counters for how often translation is invoked vs. skipped
This would help identify bottlenecks and track the value-add of the new features.
♻️ Duplicate comments (1)
echo/server/dembrane/transcribe.py (1)
60-76: 🛠️ Refactor suggestionLanguage detection + translation implementation needs hardening
Your approach for detecting and conditionally translating is solid conceptually, but needs some defensive programming:
- Missing try-except around
detect()which can throw on short/ambiguous text- Variable reuse -
responsefor both Whisper output and completion response - creates potential debugging nightmares- No temperature setting for deterministic translations
- Missing case normalization for language comparison
- No system message to improve translation quality
These are all small issues but add up to potential edge case failures.
- detected_language = detect(response["text"]) - if detected_language != language and language != "multi": - translation_prompt = render_prompt( - "translate_transcription", - str(language), - {"transcript": response["text"], "detected_language": detected_language, "desired_language": language} - ) - llm_translation_response = completion( - model=SMALL_LITELLM_MODEL, - messages=[{"role": "user", "content": translation_prompt}], - api_key=SMALL_LITELLM_API_KEY, - api_base=SMALL_LITELLM_API_BASE, - api_version=SMALL_LITELLM_API_VERSION, - ) - return llm_translation_response['choices'][0]['message']['content'] - else: - return response["text"] + transcript_text = response["text"] + + try: + detected_language = detect(transcript_text) + except Exception as e: + logger.warning(f"Language detection failed: {e}") + detected_language = language or "unknown" + + if detected_language and language and detected_language.lower() != language.lower() and language != "multi": + translation_prompt = render_prompt( + "translate_transcription", + str(language).lower(), + {"transcript": transcript_text, "detected_language": detected_language, "desired_language": language} + ) + llm_translation_response = completion( + model=SMALL_LITELLM_MODEL, + messages=[ + {"role": "system", "content": "You are a professional translator."}, + {"role": "user", "content": translation_prompt} + ], + api_key=SMALL_LITELLM_API_KEY, + api_base=SMALL_LITELLM_API_BASE, + api_version=SMALL_LITELLM_API_VERSION, + temperature=0, + ) + return llm_translation_response['choices'][0]['message']['content'] + else: + return transcript_text
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
echo/server/dembrane/transcribe.py(4 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (1)
echo/server/dembrane/transcribe.py (2)
echo/server/dembrane/s3.py (1)
get_stream_from_s3(176-180)echo/server/dembrane/prompts.py (1)
render_prompt(55-88)
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: ci-check-server
🔇 Additional comments (4)
echo/server/dembrane/transcribe.py (4)
83-87: LGTM on prompt improvements!The explicit system prompts using clear language-specific instructions are much better than the previous informal ones. This will give the whisper model clearer direction, leading to more accurate transcriptions.
116-134: Disabling previous chunk context - intentional design change?You've commented out the code that retrieves previous chunk transcripts. This is a significant change in how the system processes sequential audio. While it simplifies the flow, it might affect the context-awareness of transcriptions.
Is this an intentional design change rather than an accidental comment-out? If intentional, consider removing the code instead of commenting it out to keep the codebase clean.
172-173: Consistent with previous chunk retrieval disablingThis comment-out matches the earlier disabling of previous chunk retrieval. Makes sense for consistency.
169-169: LGTM on improved prompt structureAdding the explicit "\n\nuser: Project prompt: \n\n" prefix creates a clearer separation in the prompt structure. This should help the model distinguish between instructions and content.
transcribe.pyto detect language and translate if necessary, improving multi-language support.langdetectversion 1.0.9 to dependencies inpyproject.toml,requirements-dev.lock, andrequirements.lock.Summary by CodeRabbit
New Features
Improvements
Chores
langdetectdependency to support automatic language detection.