Add min_p #608
Labels
api: vertex-ai
Issues related to the Vertex AI API.
priority: p3
Desirable enhancement or fix. May not be included in next release.
type: feature request
‘Nice-to-have’ improvement, new feature or different behavior or design.
Is your feature request related to a problem? Please describe.
Models are often incoherent at high temperatures, which makes it difficult to find a good balance of coherency and creativity.
Describe the solution you'd like
Add the
min_p
sampler as an option inGenerateContentConfig
. See https://arxiv.org/abs/2407.01082Describe alternatives you've considered
Setting the temperature high improves creativity, but makes the output less coherent. Setting the temperature lower prevents incoherency, but this sacrifices creativity. I generally see the best results when using
min_p
(values in the 0.05-0.1 range).Additional context
This feature request is a repost of google-gemini/deprecated-generative-ai-python#680
The text was updated successfully, but these errors were encountered: