Title: Add documentation for custom dependencies in RayService and LMCache+Mooncake on Kubernetes #60035
Conversation
Blaze-DSP
force-pushed
the
feat/extra-dependency
branch
from
9e86c70 to
92cb1d8
Compare
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds valuable documentation for installing custom dependencies in RayService deployments on Kubernetes. The new sections in rayservice.md, rayserve-llm-example.md, and kv-cache-offloading.md are clear and provide useful examples for users looking to manage dependencies with args and runtime_env, including a detailed guide for LMCache and Mooncake. The cross-references in handling-dependencies.md are also a good addition. My main feedback is regarding a broken URL in the curl examples, which should be fixed to ensure users can follow the guides successfully.
| Download a basic example: | ||
|
|
||
| ```sh | ||
| curl -o ray-serve.extra-dependency.yaml https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-serve.extra-dependency.yaml |
There was a problem hiding this comment.
The URL in this curl command points to a file that doesn't seem to exist in the master branch of the ray-project/kuberay repository. This will cause the example command to fail for users. Please verify the URL and update it to a valid location.
| Download a complete example combining both approaches: | ||
|
|
||
| ```sh | ||
| curl -o ray-serve.extra-dependency.yaml https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-serve.extra-dependency.yaml |
There was a problem hiding this comment.
The URL in this curl command points to a file that doesn't seem to exist in the master branch of the ray-project/kuberay repository. This will cause the example command to fail for users. Please verify the URL and update it to a valid location.
Signed-off-by: DPatel_7 <[email protected]>
Blaze-DSP
force-pushed
the
feat/extra-dependency
branch
from
92cb1d8 to
fa2bbf5
Compare
Signed-off-by: DPatel_7 <[email protected]>
ray-gardener
bot
added
serve
kouroshHakha
left a comment
kouroshHakha
left a comment
There was a problem hiding this comment.
overall looks good. Let's make sure the full yaml service lands on kuberay repo on git and then we can cross-ref it here.
| Download a basic example: | ||
|
|
||
| ```sh | ||
| curl -o ray-serve.extra-dependency.yaml https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-serve.extra-dependency.yaml |
There was a problem hiding this comment.
is there a pr for this?
There was a problem hiding this comment.
Signed-off-by: DPatel_7 <[email protected]>
Future-Outlier
left a comment
Future-Outlier
left a comment
There was a problem hiding this comment.
cc @machichima @seanlaii @400Ping @win5923 to take a look
Future-Outlier
left a comment
Future-Outlier
left a comment
There was a problem hiding this comment.
we should build a custom image instead, since this might cause other potential problem like here
ray-project/kuberay#4362 (comment)
cc @rueian
3 participants
Description
Add documentation for installing custom dependencies in RayService deployments: