-
Notifications
You must be signed in to change notification settings - Fork 108
Remove TE v1 executor #2762
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Remove TE v1 executor #2762
Conversation
for more information, see https://pre-commit.ci
…hunder into remove-te-v1-ex
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
would there be some script and/or test to make sure there isn't unexpected regressions?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For convergence, we do compare outputs between Thunder TE executor and vanilla TE in the tests already, but regarding performance there aren't any specific tests but the benchmarks. If we were to compare performance we can do so by using benchmark_litgpt.py in something like this:
# Using TE executor
python thunder/benchmarks/benchmark_litgpt.py --model_name Llama-2-7b-hf --compile thunder --checkpoint_activations False --low_precision_mode fp8-default-te --use_sdpa False
# Using TE without Thunder
python thunder/benchmarks/benchmark_litgpt.py --model_name Llama-2-7b-hf --compile eager --checkpoint_activations False --low_precision_mode fp8-default-te --use_sdpa False
What does this PR do?
After the latest update to transformer_engine library(2.10) this executor has stopped working and, since the TEv2 is now bringing the same functionality and more, it is time to move on :D