Skip to content

Add RayService extra dependency example #4362

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Blaze-DSP wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Add RayService extra dependency example #4362

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0654d28
RayService Extra Dependencies
Jan 11, 2026
8401525
quoting and example config updates
Jan 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
123 changes: 123 additions & 0 deletions ray-operator/config/samples/ray-service.extra-dependency.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
apiVersion: ray.io/v1
kind: RayService
metadata:
name: ray-serve-extra-dependency
spec:
serveConfigV2: |
applications:
- name: fruit_app
import_path: fruit.deployment_graph
route_prefix: /fruit
runtime_env:
pip:
- pandas
working_dir: "https://github.com/ray-project/test_dag/archive/78b4a5da38796123d9f9ffff59bab2792a043e95.zip"
deployments:
- name: MangoStand
num_replicas: 1
max_replicas_per_node: 1
user_config:
price: 3
ray_actor_options:
num_cpus: 0.1
- name: OrangeStand
num_replicas: 1
user_config:
price: 2
ray_actor_options:
num_cpus: 0.1
- name: PearStand
num_replicas: 1
user_config:
price: 1
ray_actor_options:
num_cpus: 0.1
- name: FruitMarket
num_replicas: 1
ray_actor_options:
num_cpus: 0.1
- name: math_app
import_path: conditional_dag.serve_dag
route_prefix: /calc
runtime_env:
pip:
- numpy
working_dir: "https://github.com/ray-project/test_dag/archive/78b4a5da38796123d9f9ffff59bab2792a043e95.zip"
deployments:
- name: Adder
num_replicas: 1
user_config:
increment: 3
ray_actor_options:
num_cpus: 0.1
- name: Multiplier
num_replicas: 1
user_config:
factor: 5
ray_actor_options:
num_cpus: 0.1
- name: Router
num_replicas: 1
rayClusterConfig:
rayVersion: "2.53.0"
headGroupSpec:
rayStartParams:
num-cpus: "0"
dashboard-host: "0.0.0.0"
template:
spec:
containers:
- name: ray-head
image: rayproject/ray:2.53.0
ports:
- containerPort: 8000
name: serve
protocol: TCP
- containerPort: 8080
name: metrics
protocol: TCP
- containerPort: 6379
name: gcs
protocol: TCP
- containerPort: 8265
name: dashboard
protocol: TCP
- containerPort: 10001
name: client
protocol: TCP
resources:
limits:
cpu: "2"
memory: "5Gi"
requests:
cpu: "2"
memory: "5Gi"
workerGroupSpecs:
- groupName: worker-group
replicas: 1
minReplicas: 1
maxReplicas: 1
numOfHosts: 1
rayStartParams: {}
template:
spec:
containers:
- name: ray-worker
image: rayproject/ray:2.53.0
# Shared dependencies via args
# System packages and pip packages installed here are accessible
# to ALL applications in this RayService
args:
- |
sudo apt-get update && \
sudo apt-get install -y --no-install-recommends curl && \
sudo rm -rf /var/lib/apt/lists/* && \
pip install httpx \
# Make sure to add the \ at the end of the last line
Comment on lines +107 to +116
Copy link
Member

@Future-Outlier Future-Outlier Jan 12, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, thanks for the PR!
Using args for dependency installation might be risky.
Since args executes first, if the installation takes > 10 mins, the liveness probe will fail and restart the Pod.
Would postStart be a better approach here?

Suggested change
# Shared dependencies via args
# System packages and pip packages installed here are accessible
# to ALL applications in this RayService
args:
- |
sudo apt-get update && \
sudo apt-get install -y --no-install-recommends curl && \
sudo rm -rf /var/lib/apt/lists/* && \
pip install httpx \
# Make sure to add the \ at the end of the last line
# Shared dependencies via lifecycle postStart hook
# System packages and pip packages installed here are accessible
# to ALL applications in this RayService
lifecycle:
postStart:
exec:
command:
- /bin/bash
- -c
- |
sudo apt-get update && \
sudo apt-get install -y --no-install-recommends curl && \
sudo rm -rf /var/lib/apt/lists/* && \
pip install httpx

Copy link
Contributor Author

@Blaze-DSP Blaze-DSP Jan 13, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is better to do it before ray start, cz if it happens after ray and takes a long time, then the serving will fail incase, dependencies are not present when serve applications starts. I did try with bigger dependencies and it seemed to work for me.

ur thoughts on this?

Copy link
Member

@Future-Outlier Future-Outlier Jan 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is a good idea because
when installation > 10 min, kubelet will restart the pod, and we will repeat it again and again

Copy link
Member

@Future-Outlier Future-Outlier Jan 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @win5923 @seanlaii to take a look

Copy link
Member

@Future-Outlier Future-Outlier Jan 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @rueian to discuss when you have time

resources:
limits:
cpu: "2"
memory: "4Gi"
requests:
cpu: "2"
memory: "4Gi"
Loading