[Bug]: Gemini returns logprobs, but not in OpenAI format

[emk]

What happened?

When calling LiteLLM using a Gemini model and the Rust async-openai client, I have just started getting the following error:

invalid type: floating point `-0.005895897917802447`, expected struct ChatChoiceLogprobs

The LiteLLM response contains logprobs values which look like:

"logprobs":-0.003003644641441635

But according to the OpenAI API docs, this should look like:

"logprobs":  { "content": [...], "refusal": [...] }

I believe this behavior changed in #9713. Logprobs have apparently been removed from the API before, when they didn't follow the OpenAI format (see #579).

I would recommend that we either follow the OpenAI API here, or not return logprobs at all.

Thank you for such a fantastic proxy!

Relevant log output

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

litellm-proxy Docker image 44f4b5172e6c

Twitter / LinkedIn details

No response

[emk]

Downgrading to v1.65.0-stable definitely fixes this.

[krrishdholakia]

Thanks for the ticket @emk

Picking this up today.

[krrishdholakia]

This change needs to be reverted, as the ability to get logprobs when doing logprobs=True is already supported.