Add more models to default pack [Ongoing]

[scosman]

We like to have a well-tested default pack of models (each model-provider pair is tested, including testing that structured output works reliably).

What models should we add next? 👍 a model comment for support, and I'll prioritize by this list.

See the separate issue tracking end-user option to add models.

[scosman]

✅ Request: Qwen 2.5

[scosman]

✅ Llama 3.3 70b

[scosman]

✅ Deepseek V3

[leonardmq]

Adding Google as a provider would be a nice one as Gemini 2.0 is coming out to prod in the next few weeks

[scosman]

@leonardmq I can add 2.0 when it's out. Is there a reason to want direct Google/Vertex access instead of Gemini via OpenRouter?

[leonardmq]

@leonardmq I can add 2.0 when it's out. Is there a reason to want direct Google/Vertex access instead of Gemini via OpenRouter?

My assumption was that having Google/Vertex as a provider would enable adding support for fine-tuning for models from Google/Vertex, similar to how Kiln currently supports fine-tuning for OpenAI models. From what I have seen, it seems OpenRouter does not support submitting fine-tuning jobs, though I may have missed some functionality there.

My primary use case right now involves creating synthetic datasets and fine-tuning models for specific tasks to then identify the best-performing and most cost-effective one - and Gemini 2.0 Flash is likely to be quite relevant there so would be great if we could fine-tune as easily as we can OpenAI models

[scosman]

Ah, fine tuning support should be its own issue (feel free to create it). Fine tuning is a lot more work than just a model addition which is a few lines of config. You're correct OpenRouter is only for inference. I need to look at the Vertex docs; I know they have some API driven fine-tuning, but need to look at their serverless hosting options. In the meantime, just export a JSONL and use their UI?

[scosman]

Deepseek R1 - I’ll add this when I’m back