Skip to content

Dedup pod killer svc and separate nodeSelectors #106

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
smirco-zoox wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Dedup pod killer svc and separate nodeSelectors #106

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions csi-driver-templates/templates/pods/_quobyte_csi_controller_pod.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,9 @@ spec:
app: quobyte-csi-controller-{{ .Values.quobyte.csiProvisionerName | replace "." "-" }}
role: quobyte-csi-{{ .Values.quobyte.csiProvisionerName | replace "." "-" }}
spec:
{{- if default "" .Values.quobyte.csiDriverNodeSelector | trim }}
{{- if .Values.quobyte.csiControllerNodeSelector }}
nodeSelector:
{{ .Values.quobyte.csiDriverNodeSelector | trim }}
{{- .Values.quobyte.csiControllerNodeSelector | toYaml | nindent 8 }}
{{- end }}
priorityClassName: system-cluster-critical
serviceAccount: quobyte-csi-controller-sa-{{ .Values.quobyte.csiProvisionerName | replace "." "-" }}
Expand Down
4 changes: 2 additions & 2 deletions csi-driver-templates/templates/pods/_quobyte_csi_node_driver_pod.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@ spec:
app: quobyte-csi-node-{{ .Values.quobyte.csiProvisionerName | replace "." "-" }}
role: quobyte-csi
spec:
{{- if default "" .Values.quobyte.csiDriverNodeSelector | trim }}
{{- if .Values.quobyte.csiDriverNodeSelector }}
nodeSelector:
{{ .Values.quobyte.csiDriverNodeSelector | trim }}
{{- .Values.quobyte.csiDriverNodeSelector | toYaml | nindent 8 }}
{{- end }}
priorityClassName: system-node-critical
serviceAccount: quobyte-csi-node-sa-{{ .Values.quobyte.csiProvisionerName | replace "." "-" }}
Expand Down
6 changes: 3 additions & 3 deletions csi-driver-templates/templates/pods/_quobyte_csi_pod_killer_cache.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@ spec:
app: quobyte-csi-pod-killer-cache-{{ .Values.quobyte.csiProvisionerName | replace "." "-" }}
role: quobyte-csi-pod-killer-cache-{{ .Values.quobyte.csiProvisionerName | replace "." "-" }}
spec:
{{- if default "" .Values.quobyte.podKillerCacheNodeSelector | trim }}
{{- if .Values.quobyte.podKiller.nodeSelector }}
nodeSelector:
{{ .Values.quobyte.podKillerCacheNodeSelector | trim }}
{{- .Values.quobyte.podKiller.nodeSelector | toYaml | nindent 8 }}
{{- end }}
priorityClassName: system-cluster-critical
serviceAccount: quobyte-csi-pod-killer-cache-sa-{{ .Values.quobyte.csiProvisionerName | replace "." "-" }}
Expand All @@ -37,7 +37,7 @@ spec:
apiVersion: v1
kind: Service
metadata:
name: quobyte-pod-killer-cache
name: quobyte-pod-killer-cache-{{ .Values.quobyte.csiProvisionerName | replace "." "-" }}
namespace: kube-system
spec:
selector:
Expand Down
2 changes: 1 addition & 1 deletion csi-driver-templates/templates/pods/containers/_quobyte_pod_mount_monitor_container.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
args:
- "--node_name=$(KUBE_NODE_NAME)"
- "--driver_name={{ .Values.quobyte.csiProvisionerName }}"
- "--service_url=http://quobyte-pod-killer-cache.$(NAMESPACE).svc.cluster.local:80/"
- "--service_url=http://quobyte-pod-killer-cache-{{ .Values.quobyte.csiProvisionerName | replace "." "-" }}.$(NAMESPACE).svc.cluster.local:80/"
- "--monitoring_interval={{ .Values.quobyte.podKiller.monitoringInterval }}"
- "--role=monitor"
env:
Expand Down
10 changes: 6 additions & 4 deletions csi-driver-templates/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,11 +75,12 @@ quobyte:
# Empty means all the nodes in the k8s cluster i.e. no node selector will be used.
# nodeSelector: "<node-label-name>: '<node-label-value>'"
# Example: csiDriverNodeSelector: "quobyteCsiDriverNode: 'true'"
csiDriverNodeSelector: ""
# Pod killer cache node selector - leaving it empty creates pod killer cache pod
csiDriverNodeSelector: {}
# CSI controller node selector - leaving it empty creates CSI controller pod
# on any of the k8s node
# Example: podKillerCacheNodeSelector: "quobytePodCacheNode: 'true'"
podKillerCacheNodeSelector: ""
# nodeSelector object of form: <node-label-name>: '<node-label-value>'
csiControllerNodeSelector: {}

podKiller:
# To disable pod killer, uninstall current CSI driver (helm uninstall <chart-name>)
# set enable: false and install CSI driver again
Expand All @@ -94,6 +95,7 @@ quobyte:
# pod killer pod so that the cache endpoint is resolved.
# See https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#pod-s-dns-policy
dnsPolicy: ""
nodeSelector: {}
Copy link
Collaborator

@venkatsc venkatsc Aug 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@smirco-zoox
I think, cacheNodeSelector would be more opt

Copy link
Author

@smirco-zoox smirco-zoox Aug 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why? it's json path is quobyte.podKiller.nodeSelector

I'd argue that all the other components should also have paths like this to be more consistent with how podKiller is setup.

e.g. quobyte.csiController.nodeSelector instead of quobyte.csiControllerNodeSelector

However this restructures your values quite a bit more, so is a more major release of the helm chart.

Copy link
Collaborator

@venkatsc venkatsc Aug 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why? it's json path is quobyte.podKiller.nodeSelector

Because, pod killer has two components:

  • pod mount monitor that runs along with other Quobyte CSI container in a single pod on each node with quobyte.csiControllerNodeSelector
  • pod killer cache runs as a separate pod on node with quobyte.podKiller.[cache]NodeSelector

This could create some ambiguity, so please modify the naming to avoid this ambiguity.

I will make a new release. Since it is a simple configuration change, it can be easily adapted.


# The dev configuration is intended for Quobyte Developers and internal use.
# Please do NOT change the dev: configuration unless otherwise advised to change.
Expand Down