Add option to configure securityContext on both pods and containers

[Kajot-dev]

Fixes: #1253
Fixes: #1660

Why is it needed?

With the introduction of PodSecurityStandard (see https://kubernetes.io/docs/concepts/security/pod-security-standards/) in Kubernetes, users are able to enforce stricter security policies. In addition to that security-hardened distributions of k8s (like Openshift/OKD) changes the default behavior in 4.12+ (due to the introduction of mentioned PSP and integration with existing SecurityContextConstraints) and requires more values in securityContexts to be defined.

Because of this, it's becoming impossible for users with stricter security to deploy the chart without forking it.

What is does?

• Adds option to configure securityContexts through the chart
• Does not change the default behavior (except for permission initContainer in internal database deployment, see below)
• Has a clear indication that changing the user id/fs group is not supported by official images and may break permissions
• Makes data-permissions-ensurer initContainer in internal database deploy always run as a root, since it should be able to use chmod (if uid via podSecurityContext was set to 999 before, was it even working at all?)

Note: This PR does not make the chart compatible with stricter security k8s distribution by default, but enables users to do so on their own.

If the idea gets accepted, I will also update the README.

[Kajot-dev]

Ok, I've checked and root is not necessary for chmod since it will check your permissions and elevate to root by itself.

[MinerYang]

Hi @Kajot-dev ,

Could you elaborate more why should we both have pods level and container level security contexts? Since almost all of the harbor pods only have one containers. And for registry pod , although it includes registry and registryctl containers, but both of them share one PV and same permission settings for them.

[Kajot-dev]

Hi @Kajot-dev ,

Could you elaborate more why should we both have pods level and container level security contexts? Since almost all of the harbor pods only have one containers. And for registry pod , although it includes registry and registryctl containers, but both of them share one PV and same permission settings for them.

@MinerYang Od course! Ability to modify securityContext on the Pod scope is not necessarily required. Main goal of this PR is for user to be able to comply with PSP restricted (which requires additional fields in securityContext on the Container scope, https://kubernetes.io/docs/concepts/security/pod-security-standards/#restricted). However #1673 achieves the same goal without exposing configuration to the users and automatically forcing most safe/strict security.

In other words if #1673 gets merged, I am satisfied and this PR can be closed.

EDIT: Well, PSP restricted also requires runAsNonRoot on pod security context, so the PR that I'm mentioning would also need to do that

[MinerYang]

Thanks @Kajot-dev
Will close this PR and please help to review #1695 as we discussed.