[Roadmap] Blackwell MXFP8 and NVFP4 RL training

[zianglih]

@HumansAnd

Miles

• MXFP8 & NVFP4

  • [feat] Expand mxfp8 quantization to support fine-grained layer precision config #614
  • Bump Megatron version to inlcude --te-precision-config-file #567
  • [Draft] [WIP] Support --fp8-param-gather for mxfp8 #919

• MXFP8

  • Implement Blackwell MXFP8 recipe #512
  • Support DeepSeek v3.2 #963

• NVFP4

  • Implement nvfp4 #546
  • Add nvfp4 quantizer files #907
  • Use TE NVFP4 reference quantizer and add bitwise exact tests #1054
  • Use FlashInfer nvfp4 quantizer
  • Avoid QDQ during weight sync and directly use TE data

SGLang

• MXFP8 & NVFP4
  • [FlashInfer v0.6.6][RL] Support fp8-last-n-bf16 RL for flashinfer_trtllm_routed moe backend sgl-project/sglang#20214
• MXFP8
  • Add mxfp8 support for online quantization, Triton dense linear, and CUTLASS MoE sgl-project/sglang#17449
  • [RL] Support per-layer mixed FP8/BF16 serving for FP8 checkpoints sgl-project/sglang#18742
  • Expand deep_gemm entrypoint to support more FP8 recipes. sgl-project/sglang#17294
  • Add DeepGEMM MXFP8 MoE backend sgl-project/sglang#26342
  • [FlashInfer v0.6.4] [RL] Integrate FlashInfer mxfp8 gemm, MoE, and routed MoE sgl-project/sglang#19537
  • [RL] Support mxfp8 DeepSeek V3 sgl-project/sglang#21280
  • [FlashInver v0.6.7] Integrate flashinfer_trtllm mxfp8 gemm sgl-project/sglang#21576
  • [RL] Fix weight update for mxfp8 flashinfer_cutlass gemm backend sgl-project/sglang#22484
  • [RL] Fix FP8 skip matching for trailing-dot prefixes sgl-project/sglang#26287
  • [RL] Fix FlashInfer TRTLLM MXFP8 dense weight layout sgl-project/sglang#28459
• NVFP4
  • [RL] Add an nvfp4 online input scale mode sgl-project/sglang#18012
  • Fix nvfp4 weight update sgl-project/sglang#18085
  • [RL] Refactor NVFP4 shuffling/swizzling to in-place replacement sgl-project/sglang#22204
  • [FlashInfer v0.6.11] [RL] Support FlashInfer per-token NVFP4 MoE sgl-project/sglang#22918

TransformerEngine

• MXFP8 & NVFP4
  • Add NVTE_BACKWARD_OVERRIDE=high_precision|dequantized NVIDIA/TransformerEngine#2644
  • Optimizations for MXFP8/NVFP4 dequantize kernels NVIDIA/TransformerEngine#2865
• NVFP4
  • Implement row-scaled NVFP4 fprop recipe NVIDIA/TransformerEngine#2931
  • Implement 4over6 NVFP4 recipe NVIDIA/TransformerEngine#2972
  • Add row-scale support to grouped GEMM quant NVIDIA/cudnn-frontend#251
  • Use cuDNN for row-scaled NVFP4 grouped GEMM NVIDIA/TransformerEngine#3042

FlashInfer

• MXFP8 & NVFP4
  • Make cute dsl mxfp8/nvfp4 quantizer bitwise exact flashinfer-ai/flashinfer#3387
• MXFP8
  • Implement cutlass_fused_moe mxfp8 flashinfer-ai/flashinfer#2581
• NVFP4
  • [feat] Trtllm-gen Per-token Nvfp4 MoE flashinfer-ai/flashinfer#3027
  • Support 4over6 nvfp4 for quantizer and fused MoE flashinfer-ai/flashinfer#3264
  • Add CuTe DSL NVFP4 quantization with 4over6 FP16 scoring flashinfer-ai/flashinfer#3448

[hanxdmech-ship-it]

Great work! We're also working on something similar and would love to collaborate and exchange ideas. Looking forward to potential synergies between our projects.