[Task][Megatron] 3.2: Distributed checkpoint compatibility validation

[lvyufeng]

Parent Issue

Part of #206 — Megatron Roadmap, Phase 3 (Megatron Core Execution MVP): Task 3.2

Description

Validate save/load behavior against Megatron checkpoint format expectations.

Requirements

• Save distributed checkpoint from candle Megatron training
• Load distributed checkpoint and resume training
• Checkpoint format compatible with Megatron conventions
• TP resharding on load (stretch — save TP=2, load TP=4)
• Optimizer state is correctly saved/loaded per rank

Blocked By

• [Task][Megatron] 3.1: NPU-first end-to-end Megatron training smoke test #259 (E2E smoke test)

Blocks

• Training resumability and fault tolerance