New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

不知能否提供Sparse Upcycling的config #2

Open

1 task

sharkdrop opened this issue Jan 22, 2025 · 0 comments

Labels

sharkdrop commented Jan 22, 2025

Describe the feature

@Adlith 有幸拜读到您的文章，很受到启发，不知道能否分享一下消融实验Table 3对应的Sparse Upcycling的config，例如MoE Layers, Core Expert Number, Universal Expert Number.

Will you implement it?

I would like to implement this feature and create a PR!

sharkdrop added the enhancement label

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment