UR error during ComfyUI's tutorial with Stable Diffusion model on Ubuntu with Intel GPU TigerLake-H GT1

[ValentinaGalataTNG]

Your question

Hello,

I get an error, UR error, when trying to run ComfyUI's tutorial with the Stable Diffusion model.

I have checked that Pytorch can use my GPU. E.g., the following Python code works just fine:

import torch
torch.manual_seed(0)
x = torch.rand(47000, 47000, dtype=torch.float32, device='xpu')

touch I have to set UR_L0_ENABLE_RELAXED_ALLOCATION_LIMITS=1 to be able to allocate >= 4Gb.

Running torch.xpu.get_device_capability() gives

{'driver_version': '1.3.29735+27', 'gpu_eu_count': 32, 'gpu_subslice_count': 2, 'has_atomic64': True, 'has_bfloat16_conversions': False, 'has_fp16': True, 'has_fp64': False, 'has_subgroup_2d_block_io': False, 'has_subgroup_matrix_multiply_accumulate': False, 'has_subgroup_matrix_multiply_accumulate_tensor_float32': False, 'max_compute_units': 32, 'max_num_sub_groups': 64, 'max_work_group_size': 512, 'name': 'Intel(R) UHD Graphics', 'platform_name': 'Intel(R) oneAPI Unified Runtime over Level-Zero', 'sub_group_sizes': [8, 16, 32], 'total_memory': 30887309312, 'type': 'gpu', 'vendor': 'Intel(R) Corporation', 'version': '12.0.0'}

The error seems to indicate an incompatibility with my GPU, but I am not sure what else I can try to fix it. I have already checked to install the required drivers and dependencies, and have also reported the issue at intel/intel-extension-for-pytorch#781.

Any tips would be very much appreciated.

Logs

Checkpoint files will always be loaded safely.
[W204 15:12:02.899304886 OperatorEntry.cpp:155] Warning: Warning only once for all operators,  other operators may also be overridden.
  Overriding a previously registered kernel for the same operator and the same dispatch key
  operator: aten::_cummax_helper(Tensor self, Tensor(a!) values, Tensor(b!) indices, int dim) -> ()
    registered at /build/pytorch/build/aten/src/ATen/RegisterSchema.cpp:6
  dispatch key: XPU
  previous kernel: registered at /build/pytorch/build/aten/src/ATen/RegisterCPU.cpp:30476
       new kernel: registered at /build/intel-pytorch-extension/build/Release/csrc/gpu/csrc/aten/generated/ATen/RegisterXPU.cpp:2971 (function operator())
Total VRAM 29456 MB, total RAM 31815 MB
pytorch version: 2.5.1+cxx11.abi
Set vram state to: NORMAL_VRAM
Device: xpu
Using pytorch attention
ComfyUI version: 0.3.13
[Prompt Server] web root: ComfyUI/web

Import times for custom nodes:
   0.0 seconds: ComfyUI/custom_nodes/websocket_image_save.py

Starting server

To see the GUI go to: http://127.0.0.1:8188
got prompt
model weight dtype torch.float16, manual cast: None
model_type EPS
Using pytorch attention in VAE
Using pytorch attention in VAE
VAE load device: xpu:0, offload device: cpu, dtype: torch.bfloat16
CLIP/text encoder model load device: xpu:0, offload device: cpu, current: cpu, dtype: torch.float16
loaded diffusion model directly to GPU
Requested to load BaseModel
loaded completely 9.5367431640625e+25 1639.406135559082 True
2025-02-04 15:12:09,982 - _logger.py - IPEX - INFO - Currently split master weight for xpu only support sgd
2025-02-04 15:12:10,002 - _logger.py - IPEX - INFO - Conv BatchNorm folding failed during the optimize process.
2025-02-04 15:12:10,021 - _logger.py - IPEX - INFO - Linear BatchNorm folding failed during the optimize process.
Requested to load SD1ClipModel
loaded completely 26585.26240234375 235.84423828125 True
2025-02-04 15:12:10,206 - _logger.py - IPEX - INFO - Currently split master weight for xpu only support sgd
2025-02-04 15:12:10,211 - _logger.py - IPEX - INFO - Conv BatchNorm folding failed during the optimize process.
2025-02-04 15:12:10,214 - _logger.py - IPEX - INFO - Linear BatchNorm folding failed during the optimize process.
!!! Exception during processing !!! UR error
Traceback (most recent call last):
  File "ComfyUI/execution.py", line 327, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "ComfyUI/execution.py", line 202, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "ComfyUI/execution.py", line 174, in _map_node_over_list
    process_inputs(input_dict, i)
  File "ComfyUI/execution.py", line 163, in process_inputs
    results.append(getattr(obj, func)(**inputs))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "ComfyUI/nodes.py", line 69, in encode
    return (clip.encode_from_tokens_scheduled(tokens), )
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "ComfyUI/comfy/sd.py", line 148, in encode_from_tokens_scheduled
    pooled_dict = self.encode_from_tokens(tokens, return_pooled=return_pooled, return_dict=True)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "ComfyUI/comfy/sd.py", line 210, in encode_from_tokens
    o = self.cond_stage_model.encode_token_weights(tokens)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "ComfyUI/comfy/sd1_clip.py", line 635, in encode_token_weights
    out = getattr(self, self.clip).encode_token_weights(token_weight_pairs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "ComfyUI/comfy/sd1_clip.py", line 45, in encode_token_weights
    o = self.encode(to_encode)
        ^^^^^^^^^^^^^^^^^^^^^^
  File "ComfyUI/comfy/sd1_clip.py", line 252, in encode
    return self(tokens)
           ^^^^^^^^^^^^
  File "ComfyUI/comfyui-venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "ComfyUI/comfyui-venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "ComfyUI/comfy/sd1_clip.py", line 224, in forward
    outputs = self.transformer(tokens, attention_mask_model, intermediate_output=self.layer_idx, final_layer_norm_intermediate=self.layer_norm_hidden_state, dtype=torch.float32)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "ComfyUI/comfyui-venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "ComfyUI/comfyui-venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "ComfyUI/comfy/clip_model.py", line 137, in forward
    x = self.text_model(*args, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "ComfyUI/comfyui-venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "ComfyUI/comfyui-venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "ComfyUI/comfy/clip_model.py", line 101, in forward
    x = self.embeddings(input_tokens, dtype=dtype)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "ComfyUI/comfyui-venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "ComfyUI/comfyui-venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "ComfyUI/comfy/clip_model.py", line 82, in forward
    return self.token_embedding(input_tokens, out_dtype=dtype) + comfy.ops.cast_to(self.position_embedding.weight, dtype=dtype, device=input_tokens.device)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "ComfyUI/comfyui-venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "ComfyUI/comfyui-venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "ComfyUI/comfy/ops.py", line 209, in forward
    return self.forward_comfy_cast_weights(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "ComfyUI/comfy/ops.py", line 205, in forward_comfy_cast_weights
    raise error
  File "ComfyUI/comfy/ops.py", line 201, in forward_comfy_cast_weights
    return torch.nn.functional.embedding(input, weight, self.padding_idx, self.max_norm, self.norm_type, self.scale_grad_by_freq, self.sparse).to(dtype=output_dtype)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: UR error

Other

Setup

• Ubuntu 24.04.1 LTS
• Python 3.12
• ComfyUI v0.0.2-736-g8d88bfaf
• GPU: Intel(R) UHD Graphics, TigerLake-H GT1

# based on https://dgpu-docs.intel.com/driver/client/overview.html#installing-client-gpus-on-ubuntu-desktop-24-04-lts

wget -qO - https://repositories.intel.com/gpu/intel-graphics.key | \
  sudo gpg --yes --dearmor --output /usr/share/keyrings/intel-graphics.gpg

echo "deb [arch=amd64,i386 signed-by=/usr/share/keyrings/intel-graphics.gpg] https://repositories.intel.com/gpu/ubuntu noble client" | \
  sudo tee /etc/apt/sources.list.d/intel-gpu-noble.list

sudo apt update

apt-get install -y libze-intel-gpu1 libze1 intel-opencl-icd clinfo intel-gsc
apt-get install -y libze-dev intel-ocloc
# missing in the instructions, unsure if needed
apt-get install -y intel-level-zero-gpu

After cloning the ComfyUI repository:

# python
pyenv install 3.12
pyenv local 3.12

# create virtual env
python -m venv comfyui-venv
source comfyui-venv/bin/activate
pip install --upgrade pip

# https://pytorch-extension.intel.com/installation?platform=gpu&version=v2.5.10%20xpu&os=linux/wsl2&package=pip
python -m pip install torch==2.5.1+cxx11.abi torchvision==0.20.1+cxx11.abi torchaudio==2.5.1+cxx11.abi intel-extension-for-pytorch==2.5.10+xpu oneccl_bind_pt==2.5.0+xpu --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/

# install other requirements
pip install -r requirements.txt