feat: add NodeResourcesFitPlus and ScarceResourceAvoidance plugin #2302

LY-today · 2024-12-20T09:11:25Z

What is your proposal:
The NodeResourcesFit plug-in of native k8s can only adopt a type of strategy for all resources, such as MostRequestedPriority and LeastRequestedPriority. However, in industrial practice, this design does not apply to some scenarios. For example: In AI scenarios, businesses that apply for GPUs prefer to occupy the entire GPU machine first to prevent GPU fragmentation; businesses that apply for CPU & MEM are prioritized and dispersed to non-GPU machines to prevent excessive consumption of CPU & MEM on GPU machines, resulting in real tasks of applying for GPUs. Pending due to insufficient non-GPU resources
. It is therefore hoped that both strategies can be extended to address this business need.

Why is this needed:
There are related descriptions above

Is there a suggested solution, if so, please add it:

plugin-one

config：

resources: 
  nvidia.com/gpu:
    type: MostAllocated
    weight: 2
  cpu:
    type: LeastAllocated
    weight: 1
  memory:
    type: LeastAllocated
    weight: 1

config description：

node score:

finalScoreNode = [(weight1 * resource1) + (weight2 * resource2) + … + (weightN* resourceN)] /(weight1+weight2+ … +weightN)

plugin-two

config：

resources: 
- nvidia.com/gpu

config description：

node score:

finalScoreNode = (allocatablesResourcesNum - requestsResourcesNum) * framework.MaxNodeScore / allocatablesResourcesNum

codecov · 2024-12-20T09:18:06Z

Codecov Report

Attention: Patch coverage is 70.08929% with 67 lines in your changes missing coverage. Please review.

Project coverage is 66.04%. Comparing base (a62dd49) to head (1b77ba5).
Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
...ins/noderesourcefitplus/node_resources_fit_plus.go	66.96%	26 Missing and 11 partials ⚠️
...arceresourceavoidance/scarce_resource_avoidance.go	73.21%	22 Missing and 8 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2302      +/-   ##
==========================================
+ Coverage   66.02%   66.04%   +0.02%     
==========================================
  Files         458      460       +2     
  Lines       53868    54147     +279     
==========================================
+ Hits        35564    35764     +200     
- Misses      15749    15807      +58     
- Partials     2555     2576      +21

Flag	Coverage Δ
unittests	`66.04% <70.08%> (+0.02%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

pkg/scheduler/plugins/scarceresourceavoidance/scarce_resource_avoidance.go

…lugis Signed-off-by: LY-today <[email protected]>

songchh · 2024-12-23T07:51:36Z

Expressing benefit, NodeResourcesFit is indeed very limited in some scenarios, especially in non-traditional CPU and Memory clusters, expensive resources like GPU do require special treatment.

pkg/scheduler/plugins/noderesourcefitplus/node_resources_fit_plus.go

Signed-off-by: LY-today <[email protected]>

ZiMengSheng · 2025-01-07T11:53:51Z

pkg/scheduler/plugins/noderesourcefitplus/node_resources_fit_plus.go

+
+func New(args runtime.Object, handle framework.Handle) (framework.Plugin, error) {
+
+	sampleArgs2, ok := args.(*config.NodeResourcesFitPlusArgs)


please rename it

ZiMengSheng · 2025-01-07T11:56:18Z

pkg/scheduler/plugins/noderesourcefitplus/node_resources_fit_plus.go

+		return 0, framework.NewStatus(framework.Error, fmt.Sprintf("get State node %q from PreScore: %v", nodeName, err))
+	}
+
+	podRequest, _ := fitsRequest(scoreState.Resource, nodeInfo)


why does it mean fitsRequest and why we need it

refer to：k8s.io/[email protected]/pkg/scheduler/framework/plugins/noderesources/fit.go:fitsRequest

Resource types used to sense pods and nodes

ZiMengSheng · 2025-01-07T12:00:29Z

pkg/scheduler/plugins/noderesourcefitplus/node_resources_fit_plus.go

+			return 0, framework.NewStatus(framework.Error, err.Error())
+		}
+
+		resourceScore, status := fit.(framework.ScorePlugin).Score(ctx, state, p, nodeName)


Do you mean to suggest duplicating MostAllocated and LeastAllocated's implementation of Score? Rather than suggesting this form of encapsulating score?

The purpose of encapsulation here is to reduce the implementation of repeated logic. Don't quite understand the advantages of copying existing logic?

ZiMengSheng · 2025-01-07T12:00:43Z

pkg/scheduler/plugins/scarceresourceavoidance/scarce_resource_avoidance.go

+
+func New(args runtime.Object, handle framework.Handle) (framework.Plugin, error) {
+
+	sampleArgs2, ok := args.(*config.ScarceResourceAvoidanceArgs)


please rename it

LY-today force-pushed the plugins-plus branch 2 times, most recently from 4e607ff to fc5493b Compare December 20, 2024 11:12

This was referenced Dec 20, 2024

[proposal]Two better resource scheduling and allocation plugins #2298

Open

[proposal]two better resource scheduling and allocation plugins kubernetes/kubernetes#129316

Closed

sftim reviewed Dec 20, 2024

View reviewed changes

pkg/scheduler/plugins/scarceresourceavoidance/scarce_resource_avoidance.go Outdated Show resolved Hide resolved

feat: add noderesourcefitplus and scarceresourceavoidance scheduler p…

507a5b5

…lugis Signed-off-by: LY-today <[email protected]>

LY-today force-pushed the plugins-plus branch from c3328a1 to 507a5b5 Compare December 23, 2024 06:28

LY-today changed the title ~~feat: add noderesourcefitplus and scarceresourceavoidance scheduler p…~~ feat: add NodeResourcesFitPlus and ScarceResourceAvoidance plugin Dec 24, 2024

ZiMengSheng reviewed Jan 2, 2025

View reviewed changes

pkg/scheduler/plugins/noderesourcefitplus/node_resources_fit_plus.go Outdated Show resolved Hide resolved

feat: add preScore

1b77ba5

Signed-off-by: LY-today <[email protected]>

ZiMengSheng reviewed Jan 7, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add NodeResourcesFitPlus and ScarceResourceAvoidance plugin #2302

feat: add NodeResourcesFitPlus and ScarceResourceAvoidance plugin #2302

LY-today commented Dec 20, 2024 •

edited

Loading

codecov bot commented Dec 20, 2024 •

edited

Loading

songchh commented Dec 23, 2024

ZiMengSheng Jan 7, 2025

LY-today Jan 7, 2025

ZiMengSheng Jan 7, 2025

LY-today Jan 7, 2025

LY-today Jan 7, 2025 •

edited

Loading

ZiMengSheng Jan 7, 2025

LY-today Jan 7, 2025

LY-today Jan 7, 2025 •

edited

Loading

ZiMengSheng Jan 7, 2025

LY-today Jan 7, 2025


		func New(args runtime.Object, handle framework.Handle) (framework.Plugin, error) {

		sampleArgs2, ok := args.(*config.NodeResourcesFitPlusArgs)


		func New(args runtime.Object, handle framework.Handle) (framework.Plugin, error) {

		sampleArgs2, ok := args.(*config.ScarceResourceAvoidanceArgs)

feat: add NodeResourcesFitPlus and ScarceResourceAvoidance plugin #2302

Are you sure you want to change the base?

feat: add NodeResourcesFitPlus and ScarceResourceAvoidance plugin #2302

Conversation

LY-today commented Dec 20, 2024 • edited Loading

plugin-one

plugin-two

codecov bot commented Dec 20, 2024 • edited Loading

Codecov Report

songchh commented Dec 23, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LY-today Jan 7, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LY-today Jan 7, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LY-today commented Dec 20, 2024 •

edited

Loading

codecov bot commented Dec 20, 2024 •

edited

Loading

LY-today Jan 7, 2025 •

edited

Loading

LY-today Jan 7, 2025 •

edited

Loading