Skip to content

feat(shardformer): add HF5 adapters for LLaDA and Ouro - #6507

Open
kiseki-zj wants to merge 4 commits into
mainfrom
feat/support_llada_and_ouro
Open

kiseki-zj wants to merge 4 commits into
mainfrom
feat/support_llada_and_ouro

Conversation

@kiseki-zj

Copy link
Copy Markdown
Collaborator

Add isolated remote-code adapters and native nonparallel policies for LLaDA, LLaDA2.1/2.2, and Ouro. Preserve official computation while adapting HF5 construction, masks, and checkpoint metadata.

Include pinned tiny fixtures, reusable 2D1 probes, and compatibility documentation. Validation: 105 CPU FP32 tests passed with Transformers 5.17.0 and PyTorch 2.13.0.

📌 Checklist before creating the PR

  • I have created an issue for this PR for traceability
  • The title follows the standard format: [doc/gemini/tensor/...]: A concise description
  • I have added relevant tags if possible for us to better distinguish different PRs
  • I have installed pre-commit: pip install pre-commit && pre-commit install

🚨 Issue number

Link this PR to your issue with words like fixed to automatically close the linked issue upon merge

e.g. fixed #1234, closed #1234, resolved #1234

📝 What does this PR do?

Summarize your work here.
if you have any plots/diagrams/screenshots/tables, please attach them here.

💥 Checklist before requesting a review

  • I have linked my PR to an issue (instruction)
  • My issue clearly describes the problem/feature/proposal, with diagrams/charts/table/code if possible
  • I have performed a self-review of my code
  • I have added thorough tests.
  • I have added docstrings for all the functions/methods I implemented

⭐️ Do you enjoy contributing to Colossal-AI?

  • 🌝 Yes, I do.
  • 🌚 No, I don't.

Tell us more if you don't enjoy contributing to Colossal-AI.

kiseki-zj and others added 2 commits October 30, 2026 14:36
Add isolated remote-code adapters and native nonparallel policies for LLaDA, LLaDA2.1/2.2, and Ouro. Preserve official computation while adapting HF5 construction, masks, and checkpoint metadata.

Include pinned tiny fixtures, reusable 2D1 probes, and compatibility documentation. Validation: 105 CPU FP32 tests passed with Transformers 5.17.0 and PyTorch 2.13.0.
@kiseki-zj
kiseki-zj marked this pull request as ready for review November 4, 2026 07:41
@kiseki-zj
kiseki-zj requested a review from a team as a code owner November 4, 2026 07:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant