Skip to content

vllm 0.11.0 arm64 #5347

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
+8 −9
Closed

vllm 0.11.0 arm64 #5347

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2e6fa46
vllm 0.11.0 arm64
junpuf Oct 7, 2025
3e0c9f8
fix ec2 test
junpuf Oct 7, 2025
4992805
set flashinfer version back
junpuf Oct 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion release_images_general.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ release_images:
public_registry: True
5:
framework: "vllm"
version: "0.10.2"
version: "0.11.0"
arch_type: "arm64"
customer_type: "ec2"
general:
Expand Down
2 changes: 1 addition & 1 deletion test/vllm/ec2/test_artifacts/test_ec2.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ def setup_env(connection):
python3 -m venv vllm_env && \
source vllm_env/bin/activate && \
pip install --upgrade pip setuptools wheel && \
pip install numpy torch tqdm aiohttp pandas datasets pillow ray vllm==0.10.0 && \
pip install numpy torch tqdm aiohttp pandas datasets pillow ray vllm==0.11.0 && \
pip install "transformers<4.54.0"
"""
connection.run(setup_command, shell=True)
Expand Down
1 change: 0 additions & 1 deletion test/vllm/ec2/utils/run_vllm_on_arm64.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,6 @@ docker run --rm \
--entrypoint /bin/bash \
-e "HUGGING_FACE_HUB_TOKEN=$HF_TOKEN" \
-e "VLLM_WORKER_MULTIPROC_METHOD=spawn" \
-e "VLLM_USE_V1=0" \
-v /fsx/.cache/huggingface:/root/.cache/huggingface \
--gpus=all \
$DLC_IMAGE \
Expand Down
6 changes: 3 additions & 3 deletions vllm/arm64/gpu/Dockerfile.arm64
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
ARG CUDA_VERSION=12.9.0
ARG CUDA_VERSION=12.9.1
ARG IMAGE_DISTRO=ubuntu22.04
ARG PYTHON_VERSION=3.12

Expand Down Expand Up @@ -75,7 +75,7 @@ RUN git clone https://github.com/facebookresearch/xformers.git && \
FROM build-base AS build-vllm
RUN git clone https://github.com/vllm-project/vllm.git && \
cd vllm && \
git checkout v0.10.2 && \
git checkout v0.11.0 && \
git submodule sync && \
git submodule update --init --recursive -j 8 && \
MAX_JOBS=16 uv build --wheel --no-build-isolation -o /wheels
Expand Down Expand Up @@ -124,7 +124,7 @@ RUN uv pip install ninja
ARG PYTHON="python3"
LABEL maintainer="Amazon AI"
LABEL dlc_major_version="1"
ARG EFA_VERSION="1.43.2"
ARG EFA_VERSION="1.43.3"
ENV DEBIAN_FRONTEND=noninteractive \
LANG=C.UTF-8 \
LC_ALL=C.UTF-8 \
Expand Down
6 changes: 3 additions & 3 deletions vllm/buildspec-arm64.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@ account_id: &ACCOUNT_ID <set-$ACCOUNT_ID-in-environment>
prod_account_id: &PROD_ACCOUNT_ID 763104351884
region: &REGION <set-$REGION-in-environment>
framework: &FRAMEWORK vllm
version: &VERSION "0.10.2"
short_version: &SHORT_VERSION "0.10"
version: &VERSION "0.11.0"
short_version: &SHORT_VERSION "0.11"
arch_type: &ARCH_TYPE arm64
autopatch_build: "False"

Expand Down Expand Up @@ -33,7 +33,7 @@ images:
<<: *BUILD_REPOSITORY
context:
<<: *BUILD_CONTEXT
image_size_baseline: 25000
image_size_baseline: 26000
device_type: &DEVICE_TYPE gpu
cuda_version: &CUDA_VERSION cu129
python_version: &DOCKER_PYTHON_VERSION py3
Expand Down