devops: make serving benchmarks reproducible

[xiaguan]

Summary

• auto-detect the current vllm bench serve command while retaining the explicit BENCH override and vllm-bench fallback
• support local and Hugging Face datasets, including subset/split, non-streaming, and offline-cache controls
• expose fixed output length, warmups, detailed JSON, and a prebuilt PegaInfer binary
• save a Prometheus snapshot beside each benchmark result set
• pass extra vLLM server arguments as an array instead of relying on unquoted word splitting

Why

The existing helper only described random-dataset inputs and assumed a standalone vllm-bench binary. That made the Qwen3-4B + DSpark serving A/B difficult to replay with current vLLM installations, ShareGPT, and cached GSM8K, and it left speculative-decoding counters outside the result bundle.

Validation

• bash -n tools/bench/run_serving_bench.sh
• shellcheck tools/bench/run_serving_bench.sh
• prek run --files tools/bench/run_serving_bench.sh
• exercised the same command paths against Qwen3-4B + DSpark on 1x RTX 5090 with current vllm bench serve: ShareGPT and offline cached GSM8K, three interleaved A/B repetitions per dataset, 12 detailed JSON results and 12 Prometheus snapshots, zero failed requests

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7aedeaa43e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Use the Rust flags for the vllm-bench fallback

When only the standalone Rust client is installed, or BENCH=vllm-bench is specified, this shared argument array invokes it with the Python vllm bench serve flags --output-len and --input-len. The maintained Rust client’s CLI contract uses --random-input-len and --random-output-len for random datasets (.agents/skills/vllm-bench/SKILL.md:100-106), so argument parsing fails before any requests are sent and the newly advertised fallback is unusable. Construct dataset-length arguments according to the selected client.

Useful? React with 👍 / 👎.