Injecting Strimzi KafkaUser tls-external as environment variables for kafka-ui

[mattia-crypto]

So in this case, I have the following secret:

Name:         kafka-internal-listener-cert
Namespace:    kafka
Labels:       <none>
Annotations:  cert-manager.io/alt-names: *.example-internal.com
              cert-manager.io/certificate-name: kafka-internal-listener-cert
              cert-manager.io/common-name: kafka-internal-listener-cert
              cert-manager.io/ip-sans:
              cert-manager.io/issuer-group: cert-manager.io
              cert-manager.io/issuer-kind: Issuer
              cert-manager.io/issuer-name: kafka-internal-listener-issuer
              cert-manager.io/uri-sans:

Type:  kubernetes.io/tls

Data
====
tls.crt:         1249 bytes
tls.key:         1679 bytes
truststore.p12:  1178 bytes
ca.crt:          1208 bytes
keystore.p12:    3375 bytes

This is my kafka listener certificate, and as you can see, it has the required information to configure kafka-ui.

This is simply a cert-manager generated certificate chain, which I use for my strimi Kafka listeners of type laodbalancer . This means this listener is of an external type.

Because this is an external listener, I need to create a KafkaUser with authentication set to tls-external. This is because using authentication to solely tls, creates KafkaUser certificates signed by a different CA, which I do not want.

Subsequently, this means I need to inject/mount the above kafka-internal-listener-cert Secret into my kafka-ui Pod. I am doing this because I am using an internal Load Balancer.

However, I cannot use this as an existingSecret because the data in the Secret is formatted which makes them unsuitable for being stored as environment variables.
I recently saw there was a PR closed related to adding volumeMounts, volumes, initContainers and env to the helm chart. However, these appear to not work for me. Anyone got a verified example of leveraging volumeMounts, volumes, initContainers and env in the helm chart?

[mattia-crypto]

Here is how I have tested with no luck:

 - name: kafka-ui
    chart: kafka-ui/kafka-ui
    namespace: kafka
    installed: true 
    needs:
      - kafka-ui-user
    values:
      - service:
          type: ClusterIP
          port: 8080
      - tolerations:
        -
          key: kubernetes.example.com/role
          operator: Equal
          value: kafka
          effect: NoSchedule
      - volumeMounts:
          - mountPath: /opt/tls/kafka-internal-tls
            name: tls
      - volumes:
          - name: tls
            secret:
              secretName: kafka-internal-listener-cert
              items:
              - key: ca.crt
                path: ca.crt
              - key: keystore.p12
                path: keystore.p12
              - key: truststore.p12
                path: truststore.p12
              - key: tls.crt
                path: tls.crt
              - key: tls.key
                path: tls.key

[5hin0bi]

Hi everyone.
@mattia-crypto as far as I know there was no release yet after the helm chart changes via PR, how exactly you are using our helm chart?
Via helm repo or by cloning the kafka-ui repo and using the current helm chart state passing path to the helm install?

[5hin0bi]

Passing this to values.yaml works fine:

volumeMounts: [
      {   
        mountPath: /opt/tls/kafka-internal-tls,
        name: tls,
      }
]

volumes: [
  {
    name: tls,
    secret:
      {
      secretName: kafka-internal-listener-cert,
        items: [
          {
            key: ca.crt,
            path: ca.crt,
          },
        ]
      } 
  },
]

It produces the deployment like this:

# Source: kafka-ui/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: release-name-kafka-ui
  labels:
    helm.sh/chart: kafka-ui-0.0.3
    app.kubernetes.io/name: kafka-ui
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/version: "latest"
    app.kubernetes.io/managed-by: Helm
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: kafka-ui
      app.kubernetes.io/instance: release-name
  template:
    metadata:
      annotations:
        checksum/config: 3764f386e31f2bddcb74e81ab5092883c5df2dbefbc67903295b1a2e85f3f100
        checksum/secret: cd7ac3884964cc0e6ac144d69ea436cae52b875cce7b3e55c769cd4aa833f6bd
      labels:
        app.kubernetes.io/name: kafka-ui
        app.kubernetes.io/instance: release-name
    spec:
      serviceAccountName: release-name-kafka-ui
      securityContext:
        {}
      containers:
        - name: kafka-ui
          securityContext:
            {}
          image: "docker.io/provectuslabs/kafka-ui:latest"
          imagePullPolicy: IfNotPresent
          envFrom:
            - configMapRef:
                name: release-name-kafka-ui
            - secretRef:
                name: release-name-kafka-ui
          ports:
            - name: http
              containerPort: 8080
              protocol: TCP
          livenessProbe:
            httpGet:
              path: /actuator/health
              port: http
            initialDelaySeconds: 60
            periodSeconds: 30
            timeoutSeconds: 10
          readinessProbe:
            httpGet:
              path: /actuator/health
              port: http
            initialDelaySeconds: 60
            periodSeconds: 30
            timeoutSeconds: 10
          resources:
            {}
          volumeMounts:
            - mountPath: /opt/tls/kafka-internal-tls
              name: tls
      volumes:
        - name: tls
          secret:
            items:
            - key: ca.crt
              path: ca.crt
            secretName: kafka-internal-listener-cert

Please try to check your configuration again because passing vollumes and volumeMounts works fine. The result only depends on the syntax correctness.

[mattia-crypto]

@5hin0bi you are correct, it is a release thing. In my case I am using helmfile, which is release focused.

[mattia-crypto]

I will try with your example and see if that helps

[mattia-crypto]

Sadly that does not do it when using helmfile...

[ix-mha]

Hi @mattia-crypto

We are trying to deploy a kafka-ui container on our k8s environment accessing the kafka-cluster deployed by the strimzi operator. Could you give me some hints or examples what I need to set to the env variables? Until now I tried connecting to the external-bootstrap-server service with setting the KAFKA_CLUSTERS_0_PROPERTIES_SSL_TRUSTSTORE_LOCATION to a truststore containing the *.crt files from the superuser secret. However, I'm getting always a "SSL handshake failed" error.

I don't know if I'm on the right way but I think it has something todo with my truststore.

If you have any examples on a deployment.yml file that should work it would be much appreciated.

[mattia-crypto]

Hi @ix-mha this is a similar problem to what I am facing.

This is a while back now, and adding TLS for the kafka-ui is not a concern at the moment, so I don't know if this will be of much help since I have forgotten most of it.

However, if I remember correctly, the strimzi operator creates certificates which are not in the right format for kafka-ui, since it requires .p12 files / keystore / truststore, so you have to think of a way to create those. I used cert-manager and password generator operator.

Also, bear in mind, I do not use helm, but helmfile, so the format is a bit different. I tried translating what @5hin0bi proposed above, but with no luck:

volumeMounts: [
      {   
        mountPath: /opt/tls/kafka-internal-tls,
        name: tls,
      }
]

volumes: [
  {
    name: tls,
    secret:
      {
      secretName: kafka-internal-listener-cert,
        items: [
          {
            key: ca.crt,
            path: ca.crt,
          },
        ]
      } 
  },
]

and translated to something helmfile can read, and that ended up being this:

release:
  - name: kafka-ui
  - chart:  kafka-ui/kafka-ui
     values:
      - volumeMounts:
          - mountPath: /opt/tls/kafka-internal-tls
            name: tls
      - volumes:
          - name: tls
            secret:
              secretName: kafka-internal-listener-cert
              items:
              - key: ca.crt
                path: ca.crt
              - key: keystore.p12
                path: keystore.p12
              - key: truststore.p12
                path: truststore.p12
              - key: tls.crt
                path: tls.crt
              - key: tls.key
                path: tls.key     

however, I must done something incorrect since it didn't really do the trick and my Deployment still didn't have the mount in place.

If you find a solution it would be great to share it here.

[ix-mha]

Unfortunatley I couldn't figure it out. We decided to access it without TLS from kafka-ui to the kafka cluster.

[fulgas]

Seems like an old thread but here is how you can connect kafka UI with a kafkaUser using Strimizi with tls and simple authorization:

Authorization as super user:

  authorization:
    type: simple
    superUsers:
      - CN=**USER**

User Creation:

- name: **USER**
   spec:
     authentication:
       type: tls
     authorization:
       type: simple
       acls:
         - resource:
             name: '*'
             patternType: literal
             type: topic
           operation: All
         - resource:
             name: '*'
             patternType: literal
             type: group
           operation: All
         - resource:
             type: cluster
           operation: All

Kafka UI configuration:

volumes:
  - name: kafka-user-credentials
    secret:
      secretName: **USER_SECRET**
      items:
        - key: user.p12
          path: user.p12
  - name: kafka-cluster-credentials
    secret:
      secretName: **CLUSTER_SECRET**-ca-cert
      items:
        - key: ca.p12
          path: ca.p12

volumeMounts:
  - name: kafka-user-credentials
    mountPath: /var/user
    readOnly: true
  - name: kafka-cluster-credentials
    mountPath: /var/cluster
    readOnly: true

env:
  - name: KAFKA_CLUSTERS_0_PROPERTIES_SECURITY_PROTOCOL
    value: SSL
  - name: KAFKA_CLUSTERS_0_PROPERTIES_SSL_TRUSTSTORE_LOCATION
    value: /var/cluster/ca.p12
  - name: KAFKA_CLUSTERS_0_PROPERTIES_SSL_TRUSTSTORE_PASSWORD
    valueFrom:
      secretKeyRef:
        name: **CLUSTER_SECRET**-ca-cert
        key: ca.password
  - name: KAFKA_CLUSTERS_0_PROPERTIES_SSL_TRUSTSTORE_TYPE
    value: PKCS12
  - name: KAFKA_CLUSTERS_0_PROPERTIES_SSL_KEYSTORE_LOCATION
    value: /var/user/user.p12
  - name: KAFKA_CLUSTERS_0_PROPERTIES_SSL_KEYSTORE_PASSWORD
    valueFrom:
      secretKeyRef:
        name: **USER_SECRET**
        key: user.password
  - name: KAFKA_CLUSTERS_0_PROPERTIES_SSL_KEYSTORE_TYPE
    value: PKCS12

Hope it helps someone else.

cheers,

Nelson Silva