Skip to content

refactor qwen moe code, use communicator to support tp+dp #6581

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
May 26, 2025

Conversation

yizhang2077
Copy link
Collaborator

Motivation

refactor qwen2/qwen3moe tp+dp code, use communicator in #6321

Modifications

Checklist

Copy link
Collaborator

@fzyzcjy fzyzcjy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM as long as tests pass, and only a few small nits

@yizhang2077 yizhang2077 force-pushed the refactor-qwen-moe-code branch from 45c99e7 to 2604818 Compare May 25, 2025 14:05
@yizhang2077 yizhang2077 force-pushed the refactor-qwen-moe-code branch from 1e4498a to 0677533 Compare May 25, 2025 14:57
@zhyncs zhyncs merged commit 65f0913 into main May 26, 2025
1 of 36 checks passed
@zhyncs zhyncs deleted the refactor-qwen-moe-code branch May 26, 2025 06:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants