Repository navigation
[BUG]: No module named 'dropout_layer_norm' #5726
Description
Is there an existing issue for this bug?
🐛 Describe the bug
ModuleNotFoundError: No module named 'dropout_layer_norm'
[2024-05-17 03:23:11,932] torch.distributed.elastic.multiprocessing.api: [ERROR] failed (exitcode: 1) local_rank: 0 (pid: 615) of binary: /usr/bin/python
dropout_layer_norm is depreated by flash_attn ,so If any other choise ?
Environment
No response
Activity
duanjunwen commentedon Jun 11, 2024
Hi @apachemycat , would you mind sharing the version of flash_atten in your environment? I am using flash-attn==2.5.7 , looks all good. Also, you can replace dropout_layer_norm with torch.nn.functional.layer_norm
& dropout, although kernel acceleration may not supported now.
zhurunhua commentedon Jul 18, 2024
watching...
JazyJiang commentedon Oct 16, 2026
Thanks for the report, and sorry for the long silence. The current
mainno longer referencesdropout_layer_norm, so this import error should not occur with an up-to-date installation. Our project requirements have also been cleaned up recently (#6456). Closing for now. If you still hit this on the latestmain, please reopen with the full traceback and yourtorch/flash-attn/ ColossalAI versions.