refactor(qwen35): split TP-shard helpers into scheduler/tp.rs - #968
Open
CAICAIIs wants to merge 10 commits into
Open
refactor(qwen35): split TP-shard helpers into scheduler/tp.rs#968CAICAIIs wants to merge 10 commits into
CAICAIIs wants to merge 10 commits into
Conversation
Split the 683-line config.rs into a config/ module with model/tokenizer/tp submodules (k3-style mod.rs entry + config/), reducing config.rs to a re-export. Behavior unchanged. cargo check + clippy --all-targets -D warnings + config tests (14/14) green. Signed-off-by: CAICAIIs <3360776475@qq.com>
…try types Rebuild the config split into a directional ownership model instead of a flat file redistribution: - model.rs: deserialize a RawConfig into a validated, TP-agnostic Config35 via TryFrom, with typed ConfigError variants for every cross-field and kernel-AOT rule. - tp.rs: validate TensorParallelConfig at construction (private fields, so invalid world_size/rank is unrepresentable) and derive a validated LocalGeometry from (Config35, TP config, cuda-graph mode). Downstream accepts LocalGeometry instead of re-deriving shards from a raw rank pair. - error.rs: typed ConfigError (thiserror) used across model + TP boundaries. - tokenizer.rs: own the frontend compatibility schema explicitly instead of hiding unread fields behind allow(dead_code). - mod.rs: explicit exports (no glob re-exports); tests co-located in the owning module, fixture + table-driven mutations. Downstream (weights, decode_buffers, batch_decode, batch_decode_graph, prefill, tp_executor) now threads LocalGeometry for all shard math. Signed-off-by: CAICAIIs <3360776475@qq.com>
… re-export recurrent.rs reads GDN_AOT_*_DIM and LINEAR_CONV_MAX_KERNEL_DIM from crate::config; re-export them from config/mod.rs. ConfigError is referenced only inside the config submodules, so the crate-level re-export is dead. Signed-off-by: CAICAIIs <3360776475@qq.com>
…nused tp() accessor The FrontendAddedToken schema is an explicitly-owned fail-closed contract whose fields exist only to shape the typed parse; attach a justified allow(dead_code) instead of letting -D warnings reject it. LocalGeometry::tp() is unused now that downstream takes LocalGeometry directly. Signed-off-by: CAICAIIs <3360776475@qq.com>
Signed-off-by: CAICAIIs <3360776475@qq.com>
…paths Expose RawConfig at the config boundary so sibling test modules can deserialize fixtures, and reference RawConfig via crate::config paths that resolve inside both the real crate and the isolated harness. Signed-off-by: CAICAIIs <3360776475@qq.com>
Reaching into the private model module for a fixture weakens the validated boundary and pulls RawConfig into the crate surface. Build the test config through the real Config35::from_file entry point with a tempdir, and stop re-exporting RawConfig (lib-only clippy flags it as unused). Signed-off-by: CAICAIIs <3360776475@qq.com>
…n_layers Signed-off-by: CAICAIIs <3360776475@qq.com>
Move the SingleGpuBackend / TpSchedulerBackend / SchedulerBackend / AsyncPrefillOutput cluster (553 lines) out of the 2624-line scheduler.rs into scheduler/backend.rs. Pure move: mod backend + use self::backend::*. Validated on A100: cargo check + cargo fmt OK, cargo test --lib 95 passed / 0 failed (7 TP tests ignored: need 2 CUDA devices + Qwen3.5 weights). scheduler.rs 2624 -> 2074 lines. Signed-off-by: CAICAIIs <3360776475@qq.com>
Move tp_prefill_items/tp_decode_items/align_prefill_results/align_decode_results/ split_decode_artifacts (170 lines) out of scheduler.rs into scheduler/tp.rs. Pure move: mod tp + use self::tp::*; pub(super) on the fns. Validated on A100: cargo check + cargo fmt OK, cargo test --lib 95 passed / 0 failed (7 TP tests ignored). scheduler.rs 2072 -> 1905. Signed-off-by: CAICAIIs <3360776475@qq.com>
CAICAIIs
force-pushed
the
refactor/qwen35-scheduler-tp-a
branch
from
August 27, 2026 05:20
dca51ba to
a9a4485
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Split the TP-shard helpers (
tp_prefill_items/tp_decode_items/align_prefill_results/align_decode_results/split_decode_artifacts) out ofscheduler.rsintoscheduler/tp.rs(k3-stylescheduler/mod.rs+scheduler/tp.rs). Behavior unchanged.Structure
Validation
cargo check— passcargo clippy --all-targets -- -D warnings— pass (matches CI)Note
Stacked on #967 (scheduler backend split) — the scheduler slices are sequential on the same
scheduler.rs. This PR's incremental change is thetp.rsextraction only.