-
Notifications
You must be signed in to change notification settings - Fork 193
Enable EPP to support endpoint discovery using pod selector #1833
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Enable EPP to support endpoint discovery using pod selector #1833
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: capri-xiyue The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
✅ Deploy Preview for gateway-api-inference-extension ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
No need to review it right now. I just made the CUJ of standalone epp work without inferencepool. Still need to fix the e2e and ut |
elevran
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cursory review to provide initial feedback (realizing this is work in progress)
The main question I have (and might be worth mentioning in the PR description) is the need for a new abstraction/type (EndPointsPool). A naive/simple solution (which perhaps does not work...) would be to copy the selector and port array into a Go InferencePool object and use datastore.PoolSet() along with disabling the Pool notification/reconciliation so it does not overwrite with nil.
Hopefully the rest of the code should not care or depend on the pool's origin (from command line or the API server)
|
assign @ahg-g for early review. |
|
assign @kfswain for early review |
Signed-off-by: Xiyue Yu <[email protected]>
| return nil | ||
| } | ||
|
|
||
| func strToUniqueIntSlice(s string) ([]int, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
there is no existing library for that in k8s?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't find existing library in k8s to change a int[] string to []int. As this is not like selector, probably it makes sense that k8s doesn't have it natively
| valueFrom: | ||
| fieldRef: | ||
| fieldPath: metadata.namespace | ||
| - name: EPP_NAME |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why do we need this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To avoid a giant PR, I kind of keep the metrics collector part unchanged in this PR.
Previously the metrics collector need inferencepool name, see https://github.com/kubernetes-sigs/gateway-api-inference-extension/blob/main/pkg/epp/metrics/collectors/inference_pool.go#L76.
As in EPP standalone mode, there is no inferencepool, I use the EPP name here.
Later I will revisit it to see whether there is better way to do it. But I think combing that in this then this PR will be too large.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
metadata.name is the name of the pod, which will look odd as it will include a hash at the end, like my-epp-xyz-abc if the deployment name was my-epp, perhaps extract the deployment name in code by dropping the hash suffix?
Signed-off-by: Xiyue Yu <[email protected]>
Signed-off-by: Xiyue Yu <[email protected]>
What type of PR is this?
/kind feature
What this PR does / why we need it:
See #1779
Which issue(s) this PR fixes:
Part of #1779
Does this PR introduce a user-facing change?: