Support hosted tools across different model providers

[jhammarstedt]

Native Tools across different model providers

One big issue with many Agent SDKs are that they don't support all tools each custom model provider offers.

E.g

• Openai has WebSearchTool(), FilesearchTool()
• Gemini has their own Tool(google_search=GoogleSearch()) and file_upload features

Building effective systems which should rely on these tools becomes hard as no provider yet supports fully swapping between models and their native tools. The solution is to build your own tools or rely on external providers for file and search which feels suboptimal and introduces complexity.

I understand the problem of trying to support google, antropic and deepseeks own tools under the same framework. But having the power to do so would very much strengthen the usability and flexibility of the Agents SDK.

[rm-openai]

Makes sense to me. I'll think about a good path forward. Let me know if you have suggestions as well?

[jhammarstedt]

Makes sense to me. I'll think about a good path forward. Let me know if you have suggestions as well?

Great, I think it would be a great value add going forward!

Just brainstorming ideas here, but one approach could be to create a GeminiClient build on the actual google-genai sdk that supports their native file upload and websearch. Then either create a standalone GeminiAgent(Agent) that implements the Agent base class methods OR add this as a configuration in model_settings when defining an agent. This behaviour could then be replicated for all the other big model providers that has their own set of tools.

Do you have any thoughts on this, or an alternative solution, happy to discuss ideas!