Usage of tools fails - RAG chatbot example

[pavelsvitek]

Hi all,

I just tried the https://sdk.vercel.ai/docs/guides/rag-chatbot example and I realized that the models have issues calling the right tools even in this very simple example.

I implemented the whole code using the documentation and I wasn't able to get the model to answer using the retrieved information from DB.

I tried switching different models and I found out that only gemini-1.5-pro-latest responded correctly.

Models which failed

• gpt-4o ❌
• gpt-4o-mini ❌
• gemini-1.5-flash-latest ❌
• gemini-1.5-pro-latest ✅

The only way I got gpt-4o to call the tools was to force toolChoice: "required" in streamText() options. But it didn't behave correctly - it didn't respond to question, it only called the tools.

Do you have similar experience?

[pavelsvitek]

Funnily enough, it works much better today after upgrading to "ai": "^4.0.22",

• gpt-4o ✅
• gpt-4o-mini ✅
• gemini-1.5-flash-latest ❌
• gemini-1.5-pro-latest ✅

Additional important note
When using streamText() one has to set maxSteps: <int> in useChat() call which sends the tool results back to API.

When using generateText({ maxSteps: <int> }) on server, this acts as agent and it loops the tool results directly on server.