Replies: 1 comment
-
|
My bad, it works. It was something upstream I was messing up. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I hosted a model with
vllm.entrypoints.openai.api_server. I tried to query it with AyncOpenAI() client instead of OpenAI() but I got the error that base_url is not a supported parameter. How do I run a large batch of requests Async using vllm?Beta Was this translation helpful? Give feedback.
All reactions