Zluda 3.8.8 cuDNN

[chain2k]

will nightly cuDNN work with gfx1150? Or I have to compile it by myself? Are there any manual about compilation?

[lshqqytiger]

This is built for gfx1150, but I can't test because I don't have gfx1150.
HIP-SDK-extension-gfx1150
You don't have to rebuild ZLUDA and use nightly build in v3.8.8 release.

[chain2k]

Great thanx, I'll test it on a week and write down the results here. So I need not nightly, but "ZLUDA-windows-rocm6-amd64.zip" 3.8.8 for HIP 6.2.4, right?

[jiangfeng79]

try to use zluda's cuDNN in comfyui,
0. copied [HIP-SDK-extension] to HIP 6.2's installation direction C:\Program Files\AMD\ROCm\6.2\bin etc

1. rename cudnn.dll from zluda's nightly build to cudnn64_8.dll
2. copy the dll to the python's venv where other zluda dlls are copied to
3. modify model loader:
   if torch.backends.cudnn.enabled: print("cuDNN is enabled by default.") else: torch.backends.cudnn.enabled = True print("cuDNN is enabled because zluda now support it.")

launch comfyui and i got the error:
Traceback (most recent call last): File "D:\ComfyUI\main.py", line 134, in <module> import comfy.utils File "D:\ComfyUI\comfy\utils.py", line 20, in <module> import torch File "C:\Users\USERNAME\AppData\Local\miniconda3\envs\sdnext\lib\site-packages\torch\__init__.py", line 132, in <module> raise err OSError: [WinError 127] The specified procedure could not be found. Error loading "C:\Users\USERNAME\AppData\Local\miniconda3\envs\sdnext\lib\site-packages\torch\lib\shm.dll" or one of its dependencies.

change the cudnn64_8.dll back to torch's default dll will recover from the error.
Python version: 3.10.11
pytorch version: 2.3.1+cu118

[jiangfeng79]

tried with --force-fp32 --fp32-text-enc cmd arguments, can see from logs:
model weight dtype torch.float32, manual cast: None
VAE load device: cuda:0, offload device: cpu, dtype: torch.float32
CLIP/text encoder model load device: cpu, offload device: cpu, current: cpu, dtype: torch.float32

sd1.5 workflow runs fine.
flux workflow runs fine
florence 2 fails the same.
ultimate sd upsacle fails the same

[chain2k]

@jiangfeng79 Did you successfully run Florence 2 on torch 2.3.1 without cuDNN? I tried to run it on FluxGym, but couldn't, and ended up switching it from GPU to CPU.

[jiangfeng79]

@chian2k, no i nvr managed to run Florence 2 with Zluda on windows platform, even with CPU. The only way it worked is with Pytorch/rocm WSL2.
I was expecting the availability of cudnn from zluda would enable it, seems still not yet.

[chain2k]

Pytorch/rocm WSL2 faster than zluda without WSL2?

[jiangfeng79]

Pytorch/rocm WSL2 faster than zluda without WSL2?

once the model is loaded by rocm/wsl2, it is fast comparatively. 5% faster than zluda/windows.
however, if i had a complex workflow wrt loading several different models, the loading time of the models turns too long; it is as if the model is not cached at all, some rocm model caching issue. Anyway, wsl2 only sees half of the system memory by default.

[chain2k]

@lshqqytiger I got an error after launch Error loading "...\Lib\site-packages\torch\lib\cudnn64_8.dll" or one of its dependencies.
I checked archive that you posted here - HIP-SDK-extension-gfx1150 there are no gfx1150 files, I can see only - gfx9.. and gfx 10.. files in /bin directory.
May be you gave me this link by mistake?

[lshqqytiger]

If you are getting The specified procedure could not be found error, see #72 (reply in thread).
The db, model files in bin directory are included by default regardless of what architectures I built for.

[chain2k]

@lshqqytiger copied cudnn.dll from zluda 3.8.8 nightly as cudnn64_9.dll, the same error with 9:
Error loading "..\Lib\site-packages\torch\lib\cudnn64_9.dll" or one of its dependencies.

[lshqqytiger]

Do you have torch 2.6.0? And did you add hip sdk bin directory in PATH? (or preload dependencies before importing torch)

[chain2k]

Yes, i fresh installed 2,6,0 torch. And used hip sdk before, all works when I leave default dll

[lshqqytiger]

I just realized that I omitted the extra dependencies.
You can find these files in v3.8.8 nightly release.

• sqlite3.dll
• bz2.dll
• boost-filesystem-blahblah.dll

Put them in hip sdk bin directory and try again.

[chain2k]

Thanx! It works, checked with Flux anf SDXL. Seems it's slower, but i need to test it side-by-side

[chain2k]

Yep, torch 2.3, zluda 3.8.8, no cuDNN - twice as fast on Flux. And it's a problem for my built-in GPU(
2.6 With cuDNN:

2.3 without cuDNN: