Repository navigation
fix(plugins/ray): stop RayJobs sharing the plugin's start-params map - #7851
Merged
pingsutw merged 1 commit intoAug 14, 2026
Merged
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #7851 +/- ##
=======================================
Coverage 57.22% 57.23%
=======================================
Files 931 931
Lines 58269 58269
=======================================
+ Hits 33347 33349 +2
Misses 21859 21859
+ Partials 3063 3061 -2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
1fanwang
force-pushed
the
1fannnw/ray-start-params-shared-config
branch
from
August 14, 2026 00:02
742a966 to
595ea52
Compare
When a task leaves RayStartParams unset, BuildResource assigns the plugin's configured defaults map directly to the local variable and then writes include-dashboard, node-ip-address and dashboard-host into it. That map belongs to the process-wide plugin config, and it is also handed to the RayJob CR, so every RayJob built from the defaults shares one map with the config and with each other. Copy the resolved parameters into the map that was already allocated, for both the head group and the worker groups. Signed-off-by: 1fanwang <1fannnw@gmail.com>
1fanwang
force-pushed
the
1fannnw/ray-start-params-shared-config
branch
from
August 14, 2026 00:18
595ea52 to
befe83f
Compare
pingsutw
approved these changes
Aug 14, 2026
pingsutw
left a comment
Member
There was a problem hiding this comment.
thank you. would you mind update the v2 plugin on main as well?
2 of 3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why are the changes needed?
A RayJob task that doesn't set
RayStartParamsgets the plugin's configured defaults.BuildResourcedoesn't copy them — it assigns the config's map to the local variable and then writesinclude-dashboard,node-ip-addressanddashboard-hoststraight into it.That map is
cfg.Defaults.HeadNode.StartParameters, owned by the process-wide plugin config. It's also what gets assigned to the CR (RayStartParams: headNodeRayStartParams). So the plugin config, and every RayJob built from the defaults, are all the same map.Two things follow. The plugin rewrites its own configuration on the first build, and any later edit to one RayJob's start params lands in every other RayJob and in the config:
Concurrent builds also write that map with no synchronisation, which is a data race on a plain Go map.
The defaults map ships non-empty (
disable-usage-stats), so thelen(...) > 0branch is taken on stock configuration. Worker groups take the same path.What changes were proposed in this pull request?
Copy the resolved start parameters into the map
BuildResourcealready allocates, instead of reassigning the variable to the caller's or the config's map. Applies to the head group and to each worker group.The
make(map[string]string)on the line above was already being discarded by the reassignment, so this just uses it.How was this patch tested?
sharedconfig_repro_test.go. The first two fail without the change:TestBuildResource_DoesNotMutateSharedConfig— one build with noRayStartParamsset, comparing the config's head and worker start parameters against a snapshot taken beforehand.TestBuildResource_RayJobsDoNotShareStartParamsMap— builds two RayJobs, writes a key into the first one's head start params, and checks the second RayJob and the plugin config are untouched.TestBuildResource_KeepsTaskSuppliedStartParams— a task supplying its own head and worker params still gets them through to the CR alongside the injected defaults.Each test pins its own config, because other tests in the package call
SetConfigwithout restoring it, which can empty the defaults and skip the branch under test.Raw output
Without the fix:
With the fix:
The wider run is green too:
Labels
fixed
Check all the applicable boxes