Skip to content

Commit 7b009aa

Browse files
committed
[Feature] Add LlamaCPP as server engine option
llama.cpp is exposing OpenAI compatible APIs and works perfectly fine with genai-bench.
1 parent 5262f69 commit 7b009aa

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

genai_bench/cli/option_groups.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -357,7 +357,7 @@ def server_options(func):
357357
func = click.option(
358358
"--server-engine",
359359
type=click.Choice(
360-
["vLLM", "SGLang", "TGI", "cohere-TensorRT", "cohere-vLLM"],
360+
["vLLM", "SGLang", "TGI", "cohere-TensorRT", "cohere-vLLM", "LlamaCPP"],
361361
case_sensitive=True,
362362
),
363363
required=False,

0 commit comments

Comments
 (0)