Conversation
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
| Field( | ||
| description="MoE LoRA strategy. 'per_projection' applies separate LoRA to each of w1/w2/w3. 'perft_e' applies a single bypass LoRA to the entire MoE block (PErFT-E).", | ||
| ), | ||
| ] = "per_projection" |
There was a problem hiding this comment.
New config fields missing CHANGELOG entry
Low Severity
Two new config fields are added to LoRAConfig in src/prime_rl/configs/trainer.py — moe_lora_mode and save_dtype — without a corresponding CHANGELOG.md update. Per the project rule, any PR modifying configuration structures (including added fields) in src/prime_rl/*/config.py must update the changelog.
Additional Locations (1)
Triggered by project rule: BugBot Instructions
| DTYPE_MAP = { | ||
| "bfloat16": torch.bfloat16, | ||
| "float32": torch.float32, | ||
| } |
There was a problem hiding this comment.
Duplicate DTYPE_MAP definitions across files
Low Severity
This PR introduces two new identical DTYPE_MAP / _DTYPE_MAP dictionaries (in ckpt.py and filesystem.py) that duplicate the existing DTYPE_MAP already defined in src/prime_rl/trainer/model.py. All three map the same two strings to the same torch dtypes. A single shared definition would avoid inconsistency risk if new dtypes are added later.
Additional Locations (1)
1 participant
Note
Medium Risk
Adds a new MoE LoRA application path (
perft_e) with custom forward/EP handling, which could affect training correctness and performance for MoE models. Also changes adapter checkpoint/broadcast serialization to cast to a configurable dtype, which may impact downstream loading expectations and numeric fidelity.Overview
Adds a new MoE LoRA strategy,
moe_lora_mode="perft_e", that applies a single bypass LoRA over the entire MoE block (PErFT-E) via the newMultiLoRAPERFTEmodule and updatesapply_lora_to_modelto select it vs the existing per-projection MoE LoRA.Introduces
LoRAConfig.save_dtypeand updates both weight checkpointing (WeightCheckpointManager.get_run_adapter_state_dict) and filesystem weight broadcast (adapter-only path) to cast LoRA adapter tensors to CPU in the configured dtype before saving.Written by Cursor Bugbot for commit 4eac31f. This will update automatically on new commits. Configure here.