New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

nn.Graph Tutorial of Pipieline Parallel #453

Open

strint opened this issue Mar 21, 2022 · 1 comment

Contributor

strint commented Mar 21, 2022

pipeline，需要完善教程，发现中兴killy，使用pipeline时，只配置了stage但是没有配置grad acc.

或者这个可以做成自动的？比如发现 stage id 配置以后，自动设置 grad acc steps from @chengtbf

chengtbf commented Mar 21, 2022

这个我跟文骁讨论过一次了，不仅是 grad acc 要自动， Checkpointing 也要自动，且不需要用户配置 stage id，只需要说我要做 pipeline 就行了。因为 stage id 很可能配错

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment