feat: add router replay(R3) for megatron engine#1207

Open

TaoZex wants to merge 119 commits into

areal-project:mainfrom

TaoZex:final_moe

Commits on Apr 18, 2026

feat: add router replay for megatron engine
bingyechen
committed
feat: fix
bingyechen
committed
feat: fix
bingyechen
committed
feat: add config for test
bingyechen
committed
feat: fix
bingyechen
committed
faet: fix
bingyechen
committed
fix(router): refactor
bingyechen
committed
fix(engine): fix routed_experts format
bingyechen
committed
fix(sglang): ban skip_tokenizer_init
bingyechen
committed
feat(math): add base config
bingyechen
committed

Commits on Apr 19, 2026

fix(math): fix config
bingyechen
committed
fix: fix skip_tokenizer_init
bingyechen
committed
feat(engine): fix optimizer
bingyechen
committed
feat(router): fix code
bingyechen
committed
fix(engine): remove
bingyechen
committed
fix: logger fix
bingyechen
committed
feat: add r3 log
bingyechen
committed
feat(validate): improve r3
bingyechen
committed
refactor(router_replay_patch): print
bingyechen
committed
fix(router): fix calculate router
bingyechen
committed
fix(ppo): add warning
bingyechen
committed

Commits on Apr 20, 2026

feat(config): fix
bingyechen
committed
fix(engine): fix forward_only
bingyechen
committed
fix(config ): set skip_tokenizer_init false
bingyechen
committed
fix(ppo): add dense count
bingyechen
committed
fix(math): fix eps_clip
bingyechen
committed
feat(math): add config
bingyechen
committed

Commits on Apr 21, 2026

fix(config): remove Instruct
bingyechen
committed
feat(R3): add logprob diff
bingyechen
committed
fix: add Moonlight-16B-A3B-Istruct
bingyechen
committed
fix(math): fix Moonlight-16B-A3B-Istruct
bingyechen
committed
fix(config): spill
bingyechen
committed
fix(router_replay): fix culen
bingyechen
committed
fix(engine): padding
bingyechen
committed

Commits on Apr 22, 2026

fix(router_replay_utils): fix padding
bingyechen
committed
feat: add log
bingyechen
committed
fix(scheduler): fix local
bingyechen
committed

Commits on Apr 23, 2026

refactor(router): fix router metric
bingyechen
committed

Commits on Apr 25, 2026

refactor(megatron_engine): later
bingyechen
committed
fix(engine): patch
bingyechen
committed
feat(math): add moonlight-16b-a3b-gsm8k-grpo
bingyechen
committed
fix(engine): fix mtp_num_layers
bingyechen
committed
fix(mcore/hf_save): fix mla param
bingyechen
committed
refactor: remove log
bingyechen
committed
fix(math): update optimizer
bingyechen
committed
fix(rollout): up max_head_offpolicyness
bingyechen
committed
fix(rollout): down max_head_offpolicyness
bingyechen
committed
fix(optimizer): lr fix
bingyechen
committed

Commits on Apr 26, 2026

fix(optimizer): lr
bingyechen
committed
feat(r3): fix r3
bingyechen
committed
fix(rollout): on policy
bingyechen
committed

Commits on Apr 27, 2026

fix(engine): fix save
bingyechen
committed

Commits on Apr 28, 2026

refactor: remove useless
bingyechen
committed
refactor(rlvr_r3_patch): remove moe infer
bingyechen
committed
feat(MoE): auto get num_moe_layers and topk
bingyechen
committed
Merge branch 'main' into final_moe
TaoZex
authored
refactor(megatron_engine): remove for run moe
bingyechen
committed
Merge branch 'final_moe' of https://github.com/TaoZex/AReaL into final_moe
bingyechen
committed
fix(engine): fix log
bingyechen
committed
feat(controller): fix log
bingyechen
committed
docs(infra): fix
bingyechen
committed
refactor(r3): fix split sample
bingyechen
committed
feat(sglang): R3 monkey patch
bingyechen
committed
refactor(launcher): remove
bingyechen
committed
feat(actor): fix for imp_weight
bingyechen
committed

Commits on Apr 29, 2026

feat(engine): add use_precision_aware_optimizer_no_fp8_or_ds_fp8(for run moe)
bingyechen
committed
refactor: fix log
bingyechen
committed
fix(guard): remove useless log
bingyechen
committed
refactor: restore local.py
bingyechen
committed
docs: remove
bingyechen
committed
style: log level
bingyechen
committed
refactor(r3): remove metric
bingyechen
committed
refactor(engine): remove RouterReplay statics
bingyechen
committed
Merge branch 'main' into final_moe
TaoZex
authored
feat: add docs and tests
bingyechen
committed
test(r3_mask_alignment): remove unused test
bingyechen
committed
test(r3_mask_alignment): fix test
bingyechen
committed
test(router_replay): fix test
bingyechen
committed
refactor(tests): get model
bingyechen
committed
fix: fix test
bingyechen
committed
test(torchrun): add prox_logp
bingyechen
committed
docs: remove useless
bingyechen
committed
Merge branch 'main' into final_moe
TaoZex
authored
docs: remove
bingyechen
committed
Merge branch 'final_moe' of https://github.com/TaoZex/AReaL into final_moe
bingyechen
committed
fix(trainer): remove useless code
bingyechen
committed

Commits on Apr 30, 2026

fix(ppo): add r3 for _compute_logp
bingyechen
committed
refactor(ppo): fix metric
bingyechen
committed
feat(ppo): add _r3_effectiveness_stats
bingyechen
committed

Commits on May 1, 2026

fix(ppo/actor): fix metric
bingyechen
committed
feat: all stage r3 log
bingyechen
committed
refactor(logging): fix log
bingyechen
committed
fix(router_replay): add mask
bingyechen
committed
fix(router_replay): zero row fix
bingyechen
committed

Commits on May 2, 2026

feat(r3): add deep log
bingyechen
committed
fix(router_replay): fix case num_sgl_tokens more than real_tokens
bingyechen
committed
fix(config): down max_concurrent_rollouts
bingyechen
committed

Commits on May 4, 2026

refactor(examples/math): pp
bingyechen
committed
fix(congig): fix
bingyechen
committed

Commits on May 5, 2026

feat(diag): pp log
bingyechen
committed

Commits on May 6, 2026

fix(router_replay): cp
bingyechen
committed
fix(engine): test
bingyechen
committed
fix(archon_utils): revert test
bingyechen
committed
fix(examples/math): test config
bingyechen
committed
feat(diag): cp log
bingyechen
committed
fix: add cp_local
bingyechen
committed
fix: config
bingyechen
committed
fix(engine): disable cp_local
bingyechen
committed
fix(examples/math): fix oom
bingyechen
committed

Commits on May 7, 2026

refactor: remove log
bingyechen
committed
refactor: remove
bingyechen
committed
refactor: remove diag
bingyechen
committed
refactor(megatron_engine): fix
bingyechen
committed
refactor(ppo): remove useless
bingyechen
committed
feat(math): fix config
bingyechen
committed
Merge branch 'main' into final_moe
TaoZex
authored
refactor(router): fix precommit
bingyechen
committed
feat: fix pre commit
root
committed
Merge branch 'main' into final_moe
TaoZex
authored