Skip to content

Add More Agent Clients (Qwen, DeepSeek, vLLM Local LLM) #55

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Flemington8 opened this issue Apr 21, 2025 · 4 comments
Closed

Add More Agent Clients (Qwen, DeepSeek, vLLM Local LLM) #55

Flemington8 opened this issue Apr 21, 2025 · 4 comments

Comments

@Flemington8
Copy link

Description
To broaden the project’s supported agent clients, I’d like to include additional options such as Qwen, DeepSeek, and potentially a locally deployed model via vLLM. These agents should also offer deeper Weights & Biases (wandb) integration for logging and analysis.

Proposed Solution

  1. Implement additional agent classes for Qwen/DeepSeek.
  2. Provide a local LLM deployment option using vLLM.
  3. Enhance wandb logging to capture important process information.

Additional Context
This expansion would help users experiment with different models in the same OpsBench environment, increasing flexibility for research or application-specific needs.

@tianyin
Copy link
Member

tianyin commented Apr 21, 2025

This is absolutely cool.

If you send out PRs, we'd love to review and merge them.

@Flemington8
Copy link
Author

This is absolutely cool.

If you send out PRs, we'd love to review and merge them.

Sure, I will release my PR about this issue soon. Hope I could contribute more idea to the community.

@Flemington8
Copy link
Author

If we simply change the url and model name in our original GTP4Turbo class, we may receive this error:

# original code
class GPT4Turbo:
    """Abstraction for OpenAI's GPT-4 Turbo model."""

    def __init__(self):
        self.cache = Cache()

    def inference(self, payload: list[dict[str, str]]) -> list[str]:
        if self.cache is not None:
            cache_result = self.cache.get_from_cache(payload)
            if cache_result is not None:
                return cache_result

        client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
        try:
            response = client.chat.completions.create(
                messages=payload,  # type: ignore
                model="gpt-4-turbo-2024-04-09",
openai.BadRequestError: Error code: 400 - {'error': {'message': 'deepseek-reasoner does not support successive user or assistant messages (messages[1] and messages[2] in your input). You should interleave the user/assistant messages in the message sequence.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}

Seem like we need to change the logic of management of trace, maybe you can refer this PR: #56
What's more, maybe this class name GPT4Turbo will be outdated soon, I suggest change to another name, like GPTClient

@HacksonClark
Copy link
Collaborator

Closed by #56

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants