Feature/Project Proposal
I would like to propose a full-duplex audio pipeline integration for the PrepPilot mock interview system. This enhancement will allow users to speak directly into their microphones and hear the AI interviewer respond with human-like voice synthesis in real-time, effectively moving the platform beyond a text-based chatbot experience.
Problem Statement
Currently, the mock interview system is entirely text-based. While this helps with formulating answers, it completely fails to simulate the pressure, timing, and conversational flow of a real technical or behavioral interview, which is a major limitation for users trying to practice their verbal communication skills.
Proposed Solution
The project could implement WebRTC or WebSocket streaming on the frontend to transmit audio chunks to the backend. The backend would then pipe these chunks through a Speech-to-Text (STT) API (like Whisper). The resulting transcription is routed to the existing LLM evaluation service, and the LLM's response is passed to a Text-to-Speech (TTS) API (like ElevenLabs) before streaming the audio bytes back to the user.
Alternatives Considered
Another possible solution is performing STT directly in the user's browser using the Web Speech API to save backend costs, but this lacks accuracy for technical jargon compared to dedicated STT models. Alternatively, we could rely on asynchronous audio file uploads, but that breaks the real-time conversational illusion.
Benefits
This would drastically improve the user experience by providing a highly immersive, stress-testing environment that closely mimics real-world interviews. It will train users on verbal pacing, reducing filler words, and thinking out loud.
Priority
None
Additional Context
No response
Checklist
Feature/Project Proposal
I would like to propose a full-duplex audio pipeline integration for the PrepPilot mock interview system. This enhancement will allow users to speak directly into their microphones and hear the AI interviewer respond with human-like voice synthesis in real-time, effectively moving the platform beyond a text-based chatbot experience.
Problem Statement
Currently, the mock interview system is entirely text-based. While this helps with formulating answers, it completely fails to simulate the pressure, timing, and conversational flow of a real technical or behavioral interview, which is a major limitation for users trying to practice their verbal communication skills.
Proposed Solution
The project could implement WebRTC or WebSocket streaming on the frontend to transmit audio chunks to the backend. The backend would then pipe these chunks through a Speech-to-Text (STT) API (like Whisper). The resulting transcription is routed to the existing LLM evaluation service, and the LLM's response is passed to a Text-to-Speech (TTS) API (like ElevenLabs) before streaming the audio bytes back to the user.
Alternatives Considered
Another possible solution is performing STT directly in the user's browser using the Web Speech API to save backend costs, but this lacks accuracy for technical jargon compared to dedicated STT models. Alternatively, we could rely on asynchronous audio file uploads, but that breaks the real-time conversational illusion.
Benefits
This would drastically improve the user experience by providing a highly immersive, stress-testing environment that closely mimics real-world interviews. It will train users on verbal pacing, reducing filler words, and thinking out loud.
Priority
None
Additional Context
No response
Checklist