AssistantResponse with TTS best approach?

[j0kerZ]

Hi everyone,

I'm trying to implement a chatbot assistant with a voice (TTS) function with Next.js and Vercel AI SDK.

Afaik, the AssistantResponse only returns as a stream to the useAssistant hook. So my current workflow is:

1. Chat send to chat API route /api/chat
2. Backend return AssistantResponse
3. useAssistant hook to check that the response is finished
4. Send the last message to voice API route /api/voice
5. The voice API sends the request to the OpenAI tts-1 model
6. Stream back the TTS response to UI

The problem with this approach is that it creates a great delay between text and voice response (I.e: when the user starts typing the next message, the voice returns)

So is there any workaround for this use case?
Such as combining these two APIs as 1 (E.g: wait for the AssistantResponse in the backend and send to the TTS model immediately)

Thank you all in advance!