Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable stopping when call (/token) budget hit #1070

Open
leondz opened this issue Jan 11, 2025 · 0 comments
Open

Enable stopping when call (/token) budget hit #1070

leondz opened this issue Jan 11, 2025 · 0 comments
Labels
architecture Architectural upgrades

Comments

@leondz
Copy link
Collaborator

leondz commented Jan 11, 2025

Summary

It should be possible to stop a run after a certain budget has been hit, defined in either input tokens, output tokens, or calls. We could do this by tracking prompts posed, their lengths, and output returned, and checking within a generator or probe whether or not budget is reached. Exceeding budget could be raised to the harness causing the run to end.

Example

Motivation

We should help users keep control over how much work garak poses to their targets

@leondz leondz added the architecture Architectural upgrades label Jan 11, 2025
@leondz leondz added this to the 25.02 Efficiency milestone Jan 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
architecture Architectural upgrades
Projects
None yet
Development

No branches or pull requests

1 participant