[MVP LLaDA2.0] Phase 9: Output correctness validation

[AlonKellner-RedHat]

Sub-Phase Issues

Phase 9 is split into two detailed sub-phases:

• #42 - Phase 9.1: Numerical Validation (Incremental Layer-by-Layer)
• #43 - Phase 9.2: E2E Evaluation Validation (lm-eval Integration)

This issue serves as the high-level orchestration and tracking parent for both sub-phases.

---

Context

Validate that LLaDA2.0 plugin outputs match reference implementations to ensure functional correctness beyond performance benchmarking.

Relationship: Builds on Phase 8 benchmarks (PR #38). Validation strategy: numerical → E2E → cross-implementation.

References:

• SGlang LLaDA2.0: https://github.com/sgl-project/sglang/blob/main/python/sglang/srt/models/llada2.py
• HuggingFace LLaDA2.0 collection

Deliverables

• Phase 9.1 ([MVP LLaDA2.0] Phase 9.1: Numerical Validation (Incremental Layer-by-Layer) #42): Numerical validation suite (tensor/logit comparison utilities), tolerance framework, layer-by-layer validation tests
• Phase 9.2 ([MVP LLaDA2.0] Phase 9.2: E2E Evaluation Validation (lm-eval Integration) #43): lm-eval integration for standard benchmarks (MMLU, HellaSwag, GSM8K, etc.), comparison tooling, evaluation reports
• Comparison reports vs SGlang and HuggingFace
• Tolerance documentation and assertion framework
• docs/PHASE9.1_NUMERICAL_VALIDATION.md and docs/PHASE9.2_LMEVAL_RESULTS.md

Acceptance Criteria

• Phase 9.1 complete: All 8 validation points pass, tolerance bounds documented, router precision validated
• Phase 9.2 complete: All 5 lm-eval tasks evaluated, results within tolerance (±1-2% categorical, ±2-3% generation)
• Results match SGlang LLaDA2.0 within documented tolerance
• Results match HuggingFace within documented tolerance (expect both to be approximately equal)
• Any discrepancies documented with root cause analysis

Out of Scope

• Performance optimization (Phase 8)
• Novel evaluation tasks beyond lm-eval standards

Dependencies

• HARD: Phase 8 complete (PR feat(phase7+8): Production LLaDA2.0 model + vLLM-native torch.compile optimization #38 merged)
• SOFT: Access to reference implementations

Implementation Contract

• Phase: 9 (orchestration for Phase 9.1 and 9.2)
• Role: Correctness validation orchestration owner
• In scope: Numerical validation ([MVP LLaDA2.0] Phase 9.1: Numerical Validation (Incremental Layer-by-Layer) #42), lm-eval ([MVP LLaDA2.0] Phase 9.2: E2E Evaluation Validation (lm-eval Integration) #43), cross-implementation comparison
• Out of scope: Performance work, new evaluation frameworks
• PR acceptance: Both sub-phases complete; validation suite merged; comparison reports documented; tolerance bounds established