pytorch README.md path to rocm updated

[sfinktah]

The path

export CMAKE_PREFIX_PATH="$(realpath ../../build/dist/rocm)"

Seems now to be

export CMAKE_PREFIX_PATH="$(realpath ../../build/base/rocm-cmake/dist/share/rocm)"

Also, some questions:

I'm trying to replicate the windows builds for gfx110x, etc by @jammm but it seems that the upstream rocm is too much of a moving target for the hipBLAS patches. I also wasn't sure what the recommended branch was, so tried checking out the exact commit the releases were made under, and then copying the .tar.gz sources over the top.

I've returned to building from this "main" (gfx1151) branch as that seems to be fine, though only has external-build files for pytorch 2.6.

The other question I have is related to a number of bash commands/scripts that appear in the build process. Are you actually using bash (cygwin or msys2 or smth else?). I have no issue with doing likewise, it's just that I'm never sure where to draw the line: e.g. use cygwin's python or windows python? cygwin's cmake or windows cmake? it seems that eventually something is going to get upset about the pathing being incompatible.

[sfinktah]

Some more notes about compiling with Windows. These may have been addressed in another branch already, but I am including them here so I don't lose them.

Step 0: Prep venv

It is highly recommended to use a virtual environment unless if in a throw-away
container/CI environment.

python -m venv .venv
source .venv/bin/activate

cmd.exe

python.exe -m venv venv
call venv\Scripts\activate
python.exe -m pip install --upgrade pip

Step 1: Preparing sources

# Checks out the most recent stable release branch of PyTorch, hipifies and
# applies patches.
./ptbuild checkout

cmd.exe

python ptbuild checkout
Set-ItemProperty -Path HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem -Name LongPathsEnabled -Value 1

then launch new shell (this is to enable long paths, otherwise you will hit the error "While running ptbuild: error: unable to create file library/src/tensor_operation_instance/gpu/gemm_multiply_multiply/device_gemm_multiply_multiply_xdl_f8_f8_bf16/device_gemm_multiply_multiply_xdl_f8_f8_bf16_mk_nk_mn_comp_default_instance.cpp: Filename too long")

Step 2: Install Deps

Python deps:

pip install -r src/requirements.txt
pip install mkl-static mkl-include

Step 3: Setup and Build

export CMAKE_PREFIX_PATH="$(realpath ../../build/base/rocm-cmake/dist/share/rocm)"
(cd src && USE_KINETO=OFF python setup.py develop)

cmd.exe

for %F in ("..\..\build\base\rocm-cmake\dist\share\rocm") do set CMAKE_PREFIX_PATH=%~fF
set USE_KINETO=OFF
:: required to avoid errors from cmake projects like protobuf that dislike newer version of cmake
set CMAKE_POLICY_VERSION_MINIMUM=3.5
cd src
python setup.py develop

Note: /cygdrive/r/rocm-TheRock/external-builds/pytorch/src/torch/csrc/jit/ir/ir.cpp needed

#define USE_ROCM

added, which probably indicates that this is not the correct branch to be using, as I'm sure you would have addressed the issue at the source already.

[sfinktah]

... for completeness, here is torchaudio and torchvision. Please note, actual wheel didn't work, so ...yeah.

And then...

Building the wheel... (will be saved to external-builds/src/dist)

cmd.exe

pip install build wheel
python -m build --wheel

Building torchaudio and vision

Anyway, you'll need to get rid of all trace of HIP and such like, suggest:

cmd.exe

set HIP_PATH=
set HIP_PATH_62=
set ROCM_HOME=
set USE_ROCM=
set USE_CUDA=
set USE_PYTORCH_QNNPACK=0

Then, assuming you have HIP installed, either remove it from your Path, or (easier), rename it on disk.
Just renaming C:\Program Files\AMD\ROCm\6.2 to C:\Program Files\AMD\ROCm\6.2 - Temporarily Disabled will do it.

You need to checkout the branches that match the pytorch you just built. To double check that it is there, and working:

cmd.exe

python -c "import torch; print(f'v{torch.__version__.split(\"+\")[0]}')"

If this is... 2.6.0a0 (which it is for me, because I'm building from the wrong
branch), then just copy what I do. Otherwise, figure it out. vision should be
v0.22.0 for pytorch 2.7.0, v0.21.0 for pytorch 2.6.0.

cmd.exe

set PYTHONPATH=R:\rocm-TheRock\external-builds\pytorch\src
cd ..\..\..\external-builds
git clone --recursive https://github.com/pytorch/audio.git torchaudio
git clone --recursive https://github.com/pytorch/vision.git torchvision

:: torchaudio
cd torchaudio
git checkout v2.6.0
pip install -r requirements.txt
python setup.py bdist_wheel

:: vision
cd ..\torchvision
git checkout v0.21.0
set DISTUTILS_USE_SDK=1
python setup.py bdist_wheel

You should now have the following wheels. I'm leaving my absolute path in (again) just for clarity.

R:\rocm-TheRock\external-builds\pytorch\src\dist\torch-2.6.0a0+git8382190-cp311-cp311-win_amd64.whl
R:\rocm-TheRock\external-builds\torchaudio\dist\torchaudio-2.6.0a0+d883142-cp311-cp311-win_amd64.whl
R:\rocm-TheRock\external-builds\torchvision\dist\torchvision-0.21.0+7af6987-cp311-cp311-win_amd64.whl

Now to check if it all works

cmd.exe

:: deactivate our development venv
%VIRTUAL_ENV%\scripts\deactivate
:: go to where jammm's wheels are installed
cd c:\zluda\comfy-rock
c:
:: (a) don't laugh at my path
:: (b) yes, you really can change directories like that, it's just super weird

:: activate our production venv
venv\Scripts\activate
pip uninstall torch torchvision torchaudio -y --quiet
pip install ^
    R:\rocm-TheRock\external-builds\pytorch\src\dist\torch-2.6.0a0+git8382190-cp311-cp311-win_amd64.whl ^
    R:\rocm-TheRock\external-builds\torchaudio\dist\torchaudio-2.6.0a0+d883142-cp311-cp311-win_amd64.whl ^
    R:\rocm-TheRock\external-builds\torchvision\dist\torchvision-0.21.0+7af6987-cp311-cp311-win_amd64.whl

Epic fail

cmd.exe

comfy --normalvram

  File "C:\zluda\comfy-rock\venv\Lib\site-packages\torch\cuda\__init__.py", line 310, in _lazy_init
    raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled

Help!!!

I manually nopped a few checks so comfyui would run, but something is NQR (not quite right). dev.type == 'hip', btw.

[sfinktah]

update

I cherrypicked some scripts, and am trying again with those.

git restore --source 5d61dbb --worktree --staged \
  build_pytorch_all.sh \
  build_pytorch_audio.sh \
  build_pytorch_torch.sh \
  build_pytorch_vision.sh \
  checkout_and_build_all.sh \
  checkout_pytorch_all.sh \
  env_init.sh \
  pytorch_audio_repo.py \
  pytorch_torch_repo.py \
  pytorch_vision_repo.py \
  repo_management.py
  
python pytorch_torch_repo.py checkout

Seems to give me 2.7.0 install scripts, but the latest patches.

[jammm]

I would highly recommend you build from TheRock main branch. These forks are not really maintained and you won't get much support from here.

Follow the instructions to build pytorch from there.
torchvision should build out of the box with FORCE_CUDA=1 and setting your CUDA_HOME and ROCM_HOME to the rocm folder that you built. torchaudio won't build out of the box on Windows and needs some work. You could try this branch, https://github.com/jammm/audio/tree/jam/v2.7.0_windows but it's another fork so be prepared to tinker around a bit. Otherwise I would wait for those changes to be upstreamed.

[sfinktah]

Hi @jammm ... I'll check it out. I feel I got close with torch 2.7.0 using 5d61dbb#diff-625c093a3a6309b1e5adae73696da1c7c693ffffc65812f3d57924ee07ff641b but apparently I didn't cherrypick all the patches and ended up without aotriton compiled in.

I am guessing this fork was all about the gfx1151?

lshqqytiger has apparently build some stand-alone wheels for older AMDs, so maybe he will sort it all out and I can go back to being lazy. I would rather like gfx1030 and gfx1100 support in one wheel. :)

[jammm]

yeah mainly for gfx1151. Though technically it could support other archs.

[sfinktah]

Totally, and it supported the gfx1100 great. It's only failings were that it didn't have LAPACK for some cpu tensor stuff used for automatic masking in Wan2GP (easily fixed), and that it got really sulky about large Conv3D's (which I believe is something of a traditional AMD thing involving lengths or widths above 512, and easily solved by using VAE Tiled Decoding).

That and CPU Text Encoding being really slow without Triton to compile CPU code. However having just read ROCm#409 I can see how you have all been working super hard, and how much everyone has contributed. It's also great to see AMD and non-AMD (and ex-AMD) people getting together to make community development possible.

btw, I also find boost wierd... but there are wonderful things in there. Though those mostly end up being mainlined into the C++ standard or extracted into things like fmt. Thankfully Microsoft vcpkg takes care of boost when I need it, and I no longer have to keep 37 versions.

Again, excellent work. I released guides on /r/comfyui for using the wheels on ComfyUI and Wan2GP, but there are only a handful of AMD regulars in there.

[sfinktah]

@jammm

We (the collective community) did some more work on this, and we have had a working 6.5 for gfx1100 and gfx1030 for a while now, with flash_attention, sage_attention, and triton. the sordid history of that development is at patientx/ComfyUI-Zluda#170 (comment) and the resultant automated script is at https://github.com/user-attachments/files/21155122/patientx-native-rocm-3.zip

Not sure if any of that would be of interest to you or scottt.

I also took some time to expand ADLX's Pybind demo into a more conventional python module https://pypi.org/project/ADLXPybind/ that can build itself from an sdist (if required) but has wheels now.

That was a necessary step to create https://pypi.org/project/pynvml-amd-windows/ which is a drop-in replacement (actually, it hijacks the pynvml module name) for pynvml in so-far as pyvnml is used in Crystool performance monitor.

As I say, I have no idea if any of this is of interest or use to you, but I figure we owe you and scottt for your work and for inspiring the rest of us to try a little harder to contribute. Hopefully some if it is of some use to you.

Obviously we also owe "other" scott and the rest of the "TheRock" crew an incalculable debt, and those at AMD who are working with the community on the ROCm project, but I'm not sure they would have any use for our small efforts.

And if you would be open to it, I might want to have chat about whether it would be practical for me to attempt port nanchaku (inference engine for 4-bit neural networks quantized with SVDQuant) to HIP.

[Ginxchan]

oh man, thanks a lot for the v6.5.0rc-pytorch-gfx110x wheels, it has been quite a decent speed up but most importantly fixed most of my vae decode issues and can fully use the vram on my 7900xt.

[sfinktah]

@Ginxchan If you have a 7900xt (which I assume is basically the same as the 7900xtx), then you'll probably (read: definately) get faster results with ZLUDA + Triton + sageattention, and with less memory use.

Though this wheel is very handy for nodes that don't work under ZLUDA (well, I've only found one, which was a stem-maker for songs, but I'm sure there's more).

Note: you can also install Triton and sageattention for this native wheel, but it will still not be as efficient. No idea why.

[Horus-p]

DivLOGs.txt
For what it is worth
After a freeze a few times: python -c "import torch; torch.tensor([1.0]).cuda(); print('Did it work?')",
I did take a look in the sys log Ubuntu 24.10, it seems that

GPU detected as amdgpu 0000:0a:00.0 (RDNA4 - gfx_v12_0)
VRAM: 16304M recognized correctly
All GPU IP blocks loaded
Kernel mode setting (KFD) initialized
Ring buffers tested and passed
GPU fully initialized: Initialized amdgpu 3.61.0
The issue happens AFTER successful initialization - when actual compute workloads start.
Critical lines:

text
amdgpu 0000:0a:00.0: amdgpu: PCIE atomic ops is not supported
This suggests Thunderbolt limitations in PCIe atomic operations, which ROCm compute heavily relies on.

The problem appears to be Thunderbolt PCIe limitations preventing proper ROCm compute operations, even though basic GPU initialization works fine.

This after installing, it BTW never worked before also with the wheel from here,

https://download.pytorch.org/whl/nightly/rocm7.1/torch-2.10.0.dev20251118%2Brocm7.1-cp311-cp311-manylinux_2_28_x86_64.whl

More digging:

Tell GitHub:
"ROCm misidentifies RX 9060 XT (RDNA4/gfx1200) as RDNA3/gfx1100, leading to architecture mismatch and system freezes. Also missing OpenCL compiler headers."

You can follow the story at
ROCm/ROCm#5657?