-
Notifications
You must be signed in to change notification settings - Fork 4
Add More Agent Clients (Qwen, DeepSeek, vLLM Local LLM) #55
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
This is absolutely cool. If you send out PRs, we'd love to review and merge them. |
Sure, I will release my PR about this issue soon. Hope I could contribute more idea to the community. |
If we simply change the url and model name in our original GTP4Turbo class, we may receive this error: # original code
class GPT4Turbo:
"""Abstraction for OpenAI's GPT-4 Turbo model."""
def __init__(self):
self.cache = Cache()
def inference(self, payload: list[dict[str, str]]) -> list[str]:
if self.cache is not None:
cache_result = self.cache.get_from_cache(payload)
if cache_result is not None:
return cache_result
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
try:
response = client.chat.completions.create(
messages=payload, # type: ignore
model="gpt-4-turbo-2024-04-09", openai.BadRequestError: Error code: 400 - {'error': {'message': 'deepseek-reasoner does not support successive user or assistant messages (messages[1] and messages[2] in your input). You should interleave the user/assistant messages in the message sequence.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}} Seem like we need to change the logic of management of trace, maybe you can refer this PR: #56 |
Closed by #56 |
Description
To broaden the project’s supported agent clients, I’d like to include additional options such as Qwen, DeepSeek, and potentially a locally deployed model via vLLM. These agents should also offer deeper Weights & Biases (wandb) integration for logging and analysis.
Proposed Solution
Additional Context
This expansion would help users experiment with different models in the same OpsBench environment, increasing flexibility for research or application-specific needs.
The text was updated successfully, but these errors were encountered: