Skip to content

Add min_p #608

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
carterprince opened this issue Apr 1, 2025 · 1 comment
Open

Add min_p #608

carterprince opened this issue Apr 1, 2025 · 1 comment
Labels
api: vertex-ai Issues related to the Vertex AI API. priority: p3 Desirable enhancement or fix. May not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@carterprince
Copy link

carterprince commented Apr 1, 2025

Is your feature request related to a problem? Please describe.
Models are often incoherent at high temperatures, which makes it difficult to find a good balance of coherency and creativity.

Describe the solution you'd like
Add the min_p sampler as an option in GenerateContentConfig. See https://arxiv.org/abs/2407.01082

Image

Describe alternatives you've considered
Setting the temperature high improves creativity, but makes the output less coherent. Setting the temperature lower prevents incoherency, but this sacrifices creativity. I generally see the best results when using min_p (values in the 0.05-0.1 range).

Additional context
This feature request is a repost of google-gemini/deprecated-generative-ai-python#680

@carterprince carterprince added priority: p3 Desirable enhancement or fix. May not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. labels Apr 1, 2025
@demoncoder-crypto
Copy link

@carterprince check this #615

@jscud jscud added the api: vertex-ai Issues related to the Vertex AI API. label Apr 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: vertex-ai Issues related to the Vertex AI API. priority: p3 Desirable enhancement or fix. May not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

No branches or pull requests

3 participants