Skip to content

[ci] Remove --runtime=nvidia from launch_container.sh #2953

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

[ci] Remove --runtime=nvidia from launch_container.sh #2953

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0523af7
[ci] Remove --runtime=nvidia from launch_container.sh
ethnzhng Nov 17, 2025
650cea4
Update tests/integration/launch_container.sh
ethnzhng Nov 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 0 additions & 9 deletions tests/integration/launch_container.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,10 +84,7 @@ if [[ "$(support_nvme)" == *"true"* ]]; then
fi

is_llm=false
if [[ "$platform" == *"-gpu"* ]]; then # if the platform has cuda capabilities
runtime="nvidia"
elif [[ "$platform" == *"lmi"* || "$platform" == *"trtllm"* || "$platform" == *"tensorrt-llm"* ]]; then # Runs multi-gpu
runtime="nvidia"
is_llm=true
if [[ "$(is_p4d_or_p5)" == *"true"* || $is_multi_node ]]; then
shm="20gb"
Expand Down Expand Up @@ -157,7 +154,6 @@ if $is_multi_node; then
-e DJL_LEADER_ADDR=${leader_hostname} \
-e DJL_WORKER_ADDR_FORMAT="${LWS_NAME}-${GROUP_INDEX}-%d.${LWS_NAME}.${NAMESPACE}" \
${env_file} \
${runtime:+--runtime="${runtime}"} \
${shm:+--shm-size="${shm}"} \
${host_device:+ ${host_device}} \
"${docker_image}" "service ssh start; djl-serving"
Expand All @@ -179,7 +175,6 @@ if $is_multi_node; then
-e AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY \
-e AWS_SESSION_TOKEN=$AWS_SESSION_TOKEN \
${env_file} \
${runtime:+--runtime="${runtime}"} \
${shm:+--shm-size="${shm}"} \
${host_device:+ ${host_device}} \
"${docker_image}" "service ssh start; /usr/bin/python3 /opt/djl/partition/run_multi_node_setup.py 2>&1 | tee /opt/djl/logs/lmi-worker.log; tail -f"
Expand Down Expand Up @@ -207,7 +202,6 @@ elif $is_sm_neo_context; then
-e AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY \
-e AWS_SESSION_TOKEN=$AWS_SESSION_TOKEN \
${env_file} \
${runtime:+--runtime="${runtime}"} \
${shm:+--shm-size="${shm}"} \
${host_device:+ ${host_device}} \
"${docker_image}"
Expand All @@ -228,7 +222,6 @@ elif $is_partition; then
-e AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID \
-e AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY \
-e AWS_SESSION_TOKEN=$AWS_SESSION_TOKEN \
${runtime:+--runtime="${runtime}"} \
${shm:+--shm-size="${shm}"} \
${host_device:+ ${host_device}} \
"${docker_image}" \
Expand All @@ -247,7 +240,6 @@ elif [[ "$docker_image" == *"text-generation-inference"* ]]; then
-e AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID \
-e AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY \
-e AWS_SESSION_TOKEN=$AWS_SESSION_TOKEN \
${runtime:+--runtime="${runtime}"} \
${shm:+--shm-size="${shm}"} \
"${docker_image}" \
${args})
Expand All @@ -272,7 +264,6 @@ else
-e AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY \
-e AWS_SESSION_TOKEN=$AWS_SESSION_TOKEN \
$uid_mapping \
Copy link
Contributor

@xyang16 xyang16 Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just realized this docker run doesn't have --gpus, we can add in line 253.

    --gpus=all \

Copy link
Contributor

@xyang16 xyang16 Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also line 184, 213, 234

${runtime:+--runtime="${runtime}"} \
${shm:+--shm-size="${shm}"} \
${host_device:+ ${host_device}} \
"${docker_image}" \
Expand Down
1 change: 1 addition & 0 deletions tests/integration/llm/prepare.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -458,6 +458,7 @@
"option.trust_remote_code": True,
"option.max_model_len": 16384,
"option.gpu_memory_utilization": "0.9",
"option.enable_expert_parallel": True,
},
"llama-4-scout-17b-16e-instruct": {
"option.model_id": "s3://djl-llm/Llama-4-Scout-17B-16E-Instruct/",
Expand Down
Loading