Fails to install on OpenShift due to explicitly assigned user ids

[pgodowski]

Describe the bug

Somewhat similar to #5 , yet the issue is that after the oeprator is installed, the StatefulSet fails to progress with creating operator Pod.

The issues are related to the fact that the operator:

• sets explicit `runAsUser value - whereas on OCP, it is OpenShift which assigned user id within the specific numeric range
• sets explicit fsGroup value - instead of defering to OCP the selection of the user id value
• init script https://github.com/yugabyte/yugabyte-k8s-operator/blob/main/chart/templates/configs.yaml#L216 launched from ConfigMap *-yugaware-pg-prerun executes chown into the specified fsGroup value, which will fail on OCP
  • again, on OCP, both runAsUser and fsGroup should be assigned automatically by OCP

To Reproduce

Install operator into OpenShift Container Platform 4.10+

Expected behavior

Operator should start without any failures

Screenshots

Failure on OCP

Init container failure, once runAsUser and fsGroup was removed manually from the operator StatefulSet definition

[amannijhawan]

@pgodowski
Did you get a chance to redeploy this with the following values set ?

rbac:
  create: false

ocpCompatibility:
  enabled: true

securityContext:
  enabled: false

[pgodowski]

Thanks Aman - yet it would still fail - see that in https://github.com/yugabyte/yugabyte-k8s-operator/blob/main/chart/templates/configs.yaml#L216 there is chown command executed which would fail even if securityContext.enabled: false is set. In general, on OCP, there is no need to deal with userIDs and fsGroup IDs, as OCP makes sure that user ids are assigned automatically by admission controller and mounted PVCs have file ownership set to match the Pod's user id.

This is true on OCP, but is not true on non-OCP kubernetes, like EKS, AKS, etc - thus I think such logic should be handled by ocpCompatibility.enabled: true flag.

BTW - I would like to have rbac.create: true, i.e. it would be ok for the helm chart to create RBAC for the operator, but it is not OK to have so many ClusterRoles, but we are having discussion about this elsewhere.

[amannijhawan]

Hi @pgodowski
Thanks again, based on original reproduction instructions
I tried the operator install with the following

helm install . -n operator-test --debug --timeout 3600s --set rbac.create=true --set yugaware.kubernetesOperatorNamespace=operator-test --set securityContext.enabled=false  --set 

I did verify that the pod was running with restricted-v2 scc

anijhawan@CYWWYVVT7L templates % oc get pod -o yaml -n operator-test | grep -i scc
      openshift.io/scc: restricted-v2

Verified that version of OC server and client > 4.10+

anijhawan@CYWWYVVT7L templates % oc version
Client Version: 4.16.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: 4.15.39
Kubernetes Version: v1.28.14+502c5ce

Version used of helm, may not matter as such.

anijhawan@CYWWYVVT7L templates % helm version
version.BuildInfo{Version:"v3.13.0", GitCommit:"825e86f6a7a38cef1112bfa606e4127a706749b1", GitTreeState:"clean", GoVersion:"go1.20.8"}

Output of get pods

anijhawan@CYWWYVVT7L templates % oc get pods -n operator-test
NAME                                       READY   STATUS    RESTARTS   AGE
chart-1734653701-yugabyte-k8s-operator-0   2/2     Running   0          8m28s