feat(qwen35): launch only on the step contract - #981
Conversation
Cut Qwen3.5 off EngineHandle/TokenEvent so ModelLine::launch returns LaunchedEngine::Stepped and Qwen35Scheduler owns submit/step/metrics. Co-authored-by: Cursor <cursoragent@cursor.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 83c128e09f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| Ok(Engine { | ||
| schedulers: vec![spawn_scheduler("qwen35-scheduler", scheduler)], |
There was a problem hiding this comment.
When CUDA context binding or decode GEMM tuning fails, spawn_scheduler returns immediately and start_with_capacity still returns Ok(Engine) before bind_if_needed runs on the driver thread. The driver then exits asynchronously, but the stepped bridge can still register and allow HTTP to bind, so startup appears successful and requests fail instead of the launch being rejected as before. Preserve a startup result channel or otherwise wait for the driver's initial bind/tune before returning the engine.
Useful? React with 👍 / 👎.
1 participant
Summary
ModelLine::launchreturnsLaunchedEngine::Steppedonly;Qwen35Schedulerimplements submit/step/metrics. This crate no longer has a dualEngineHandlepath.Engine/EngineHarness; crate-internal scheduler tests talk the contract only.PEGAINFER_TEST_MODEL_PATH=/data/models/Qwen3.5-4B—e2e_scheduler+sampling_behavior+chunked_prefill: 4 passed, 1 ignored (TP2, one GPU). e2e load aligned to 8 slots (HEAD already scheduled at 8; load was stillMAX_BATCH=64and OOM'd 16GB).-D warnings --all-targetspassed.stepof the driver thread; overlap wait stays insidestep.Handle).Test plan
cargo test --release -p pegainfer-qwen35 --features qwen35 --libcargo clippy --release -p pegainfer-qwen35 --features qwen35 --all-targets -- -D warningsPEGAINFER_TEST_MODEL_PATH=/data/models/Qwen3.5-4B:e2e_scheduler,sampling_behavior,chunked_prefillserving_tp2when 2 GPUs are available (ignored on one GPU)Made with Cursor