Skip to content

Conversation

@malsbat
Copy link

@malsbat malsbat commented Sep 25, 2025

torch.tensor fails with a ValueError when processing the islice:

(EngineCore_DP3 pid=1421) ValueError: expected sequence of length 128 at dim 1 (got 0)

Convert the islice to a list to be compatible with the PySequence_Fast functions used by pytorch (see recursive_store in pytorch/torch/csrc/utils/tensor_new.cpp).

torch.tensor fails with a ValueError when processing the islice:

  (EngineCore_DP3 pid=1421) ValueError: expected sequence of length 128 at dim 1 (got 0)

Convert the islice to a list to be compatible with the
PySequence_Fast functions used by pytorch (see recursive_store in
pytorch/torch/csrc/utils/tensor_new.cpp).
@malsbat
Copy link
Author

malsbat commented Sep 25, 2025

I found this while using the --data-parallel-size option to vllm:

python -m vllm.entrypoints.openai.api_server --port 8000 --model google/gemma-3-27b-it --served-model-name model-gemma-3-27b-it --data-parallel-size 4 --dtype bfloat16 --block-size 128 --max-model-len 16384 --max-num-seqs 128 --gpu-memory-util 0.9 --num-lookahead-slots 1 --disable-log-requests --uvicorn-log-level warning

After server is ready, issuing requests with vllm bench serve resulted in the ValueError exception mentioned above.

I am using vault.habana.ai/gaudi-docker/1.22.1/ubuntu24.04/habanalabs/pytorch-installer-2.7.1 with:

| HL-SMI Version:                              hl-1.22.1-fw-61.4.2.1          |
| Driver Version:                                     1.22.1-97ec1a4          |
| Nic Driver Version:                                 1.22.1-97ec1a4          |

@github-actions
Copy link

🚧 CI Blocked

The main CI workflow was not started for the following reason:

The DCO sign-off check has failed. Please ensure all commits are signed-off (e.g., 'git commit -s').

@github-actions
Copy link

🚧 CI Blocked

The main CI workflow was not started for the following reason:

The DCO sign-off check did not pass (status: action_required). Please ensure all commits are signed-off.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants