[Task][Megatron] 4.3: FP8 path and TransformerEngine-class integration

[lvyufeng]

Parent Issue

Part of #206 — Megatron Roadmap, Phase 4 (Advanced Features / Stretch): Task 4.3

Description

Implement FP8 compute path and TransformerEngine-style fused kernel loading for training performance.

Requirements

• FP8 dtype support (E4M3, E5M2)
• FP8 matmul kernels for NPU
• Delayed scaling / per-tensor scaling
• FP8 recipe configuration
• Extension loading mechanism for TransformerEngine-style libraries
• Fused LayerNorm + Linear kernels (stretch)

Blocked By

• Phase 3 tasks

Blocks

• Training performance parity with native frameworks