Commit 142d3ea
authored
[Feature] Add --mg-fl-prefer argument for Megatron-LM-FL vendor selection (#1183)
## Summary
- Add `--mg-fl-prefer` CLI argument to select the preferred vendor
backend (`cuda`/`musa`/`txda`) for Megatron-LM-FL plugin override
dispatch
- Sync the argument value to the `MG_FL_PREFER` environment variable at
training startup, following the same pattern as `--te-fl-prefer` /
`TE_FL_PREFER`
- Rename the argument group from "flagscale transformer engine fl" to
"flagscale fl" to reflect the broader scope
## Changes
### `flagscale/train/megatron/training/arguments_fs.py`
- Add `--mg-fl-prefer` argument with choices `['cuda', 'musa', 'txda']`,
default empty string
- Rename argument group title to "flagscale fl"
### `flagscale/train/megatron/training/training.py`
- Sync `args.mg_fl_prefer` to `os.environ['MG_FL_PREFER']` in
`pretrain()`, alongside the existing `TE_FL_PREFER` sync logic
## Usage
CLI:
```bash
python train.py --mg-fl-prefer musa --te-fl-prefer reference
```
YAML config:
```yaml
model:
mg_fl_prefer: musa
te_fl_prefer: reference
```
## Test plan
- [ ] Verify `--mg-fl-prefer musa` sets `MG_FL_PREFER=musa` in the
environment
- [ ] Verify omitting `--mg-fl-prefer` does not set `MG_FL_PREFER`
- [ ] Verify invalid values are rejected by argparse choices validation
- [ ] Verify compatibility with existing `--te-fl-prefer` argument1 parent 613b0d8 commit 142d3ea
File tree
2 files changed
+5
-1
lines changed- flagscale/train/megatron/training
2 files changed
+5
-1
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
765 | 765 | | |
766 | 766 | | |
767 | 767 | | |
768 | | - | |
| 768 | + | |
| 769 | + | |
| 770 | + | |
769 | 771 | | |
770 | 772 | | |
771 | 773 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
805 | 805 | | |
806 | 806 | | |
807 | 807 | | |
| 808 | + | |
| 809 | + | |
808 | 810 | | |
809 | 811 | | |
810 | 812 | | |
| |||
0 commit comments