[Bug]: java.net.UnknownHostException: kafka-cluster-dual-role-0.kafka-cluster-kafka-brokers.kafka.svc: nodename nor servname provided, or not known

[ChanaLii]

Bug Description

After the Kafka service deployed in EKS exposes port 9091~9093 to the public network through CLB, an exception is prompted when connecting to Kafka through the springboot client：

2025-01-07 11:24:53.952 [TID: N/A] |-INFO  [main] org.apache.kafka.common.utils.AppInfoParser [119] -| Kafka version: 3.1.2
2025-01-07 11:24:53.952 [TID: N/A] |-INFO  [main] org.apache.kafka.common.utils.AppInfoParser [120] -| Kafka commitId: f8c67dc3ae0a3265
2025-01-07 11:24:53.952 [TID: N/A] |-INFO  [main] org.apache.kafka.common.utils.AppInfoParser [121] -| Kafka startTimeMs: 1736220293950
2025-01-07 11:24:55.092 [TID: N/A] |-WARN  [kafka-admin-client-thread | adminclient-1] org.apache.kafka.clients.NetworkClient [995] -| [AdminClient clientId=adminclient-1] Error connecting to node kafka-cluster-dual-role-0.kafka-cluster-kafka-brokers.kafka.svc:9092 (id: 0 rack: null)
java.net.UnknownHostException: kafka-cluster-dual-role-0.kafka-cluster-kafka-brokers.kafka.svc: nodename nor servname provided, or not known
	at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) ~[?:?]
	at java.net.InetAddress$PlatformNameService.lookupAllHostAddr(InetAddress.java:934) ~[?:?]
	at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1543) ~[?:?]
	at java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:852) ~[?:?]
	at java.net.InetAddress.getAllByName0(InetAddress.java:1533) ~[?:?]
	at java.net.InetAddress.getAllByName(InetAddress.java:1385) ~[?:?]
	at java.net.InetAddress.getAllByName(InetAddress.java:1306) ~[?:?]
	at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27) ~[kafka-clients-3.1.2.jar:?]
	at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110) ~[kafka-clients-3.1.2.jar:?]
	at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:511) ~[kafka-clients-3.1.2.jar:?]
	at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:468) ~[kafka-clients-3.1.2.jar:?]
	at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173) ~[kafka-clients-3.1.2.jar:?]
	at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:988) ~[kafka-clients-3.1.2.jar:?]
	at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:301) ~[kafka-clients-3.1.2.jar:?]
	at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.sendEligibleCalls(KafkaAdminClient.java:1128) ~[kafka-clients-3.1.2.jar:?]
	at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.processRequests(KafkaAdminClient.java:1388) ~[kafka-clients-3.1.2.jar:?]
	at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1331) ~[kafka-clients-3.1.2.jar:?]
	at java.lang.Thread.run(Thread.java:840) ~[?:?]

Configuration in springboot：

spring:
  kafka:
    bootstrap-servers: 'abcdefg20bacda4653a371a033b1eb29d-1234567.us-east-1.elb.amazonaws.com:9092'
    consumer:
      auto-offset-reset: earliest
      group-id: test-groupId
    producer:
      value-serializer: org.springframework.kafka.support.serializer.JsonSerializer
    admin:
      properties:
        retries: 5
        retry:
          backoff:
            ms: 500
        request:
          timeout:
            ms: 30000

Steps to reproduce

1、Successfully deployed the service in EKS according to the installation instructions

Manually rename my-cluster to kafka-cluster

2、Verify that producer and consumer are running normally

kubectl -n kafka run kafka-producer -ti --image=quay.io/strimzi/kafka:0.45.0-kafka-3.9.0 --rm=true --restart=Never -- bin/kafka-console-producer.sh --bootstrap-server kafka-cluster-kafka-bootstrap:9092 --topic my-topic

kubectl -n kafka run kafka-consumer -ti --image=quay.io/strimzi/kafka:0.45.0-kafka-3.9.0 --rm=true --restart=Never -- bin/kafka-console-consumer.sh --bootstrap-server kafka-cluster-kafka-bootstrap:9092 --topic my-topic --from-beginning

3、Create a AWS-CLB to expose ports 9091~9093 to the public network

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
  name: clb-kafka-cluster-kafka-bootstrap
  namespace: kafka
spec:
  allocateLoadBalancerNodePorts: true
  externalTrafficPolicy: Cluster
  internalTrafficPolicy: Cluster
  ports:
    - name: tcp-9092
      port: 9092
      protocol: TCP
      targetPort: 9092
    - name: tcp-9091
      port: 9091
      protocol: TCP
      targetPort: 9091
    - name: tcp-9093
      port: 9093
      protocol: TCP
      targetPort: 9093
  selector:
    strimzi.io/broker-role: 'true'
    strimzi.io/cluster: kafka-cluster
    strimzi.io/kind: Kafka
    strimzi.io/name: kafka-cluster-kafka
  type: LoadBalancer

4、Springboot Application connects to kafka
The error log is as follows

java.net.UnknownHostException: kafka-cluster-dual-role-0.kafka-cluster-kafka-brokers.kafka.svc: nodename nor servname provided, or not known

Expected behavior

1. Topic is automatically created
2. Springboot producer and consumer clients can connect to Kafka

Strimzi version

0.45.0

Kubernetes version

1.31

Installation method

YAML files

Infrastructure

EKS

Configuration files and logs

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaNodePool
metadata:
  name: dual-role
  labels:
    strimzi.io/cluster: kafka-cluster
spec:
  replicas: 1
  roles:
    - controller
    - broker
  storage:
    type: jbod
    volumes:
      - id: 0
        type: persistent-claim
        size: 100Gi
        deleteClaim: false
        kraftMetadata: shared
---

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: kafka-cluster
  annotations:
    strimzi.io/node-pools: enabled
    strimzi.io/kraft: enabled
spec:
  kafka:
    version: 3.9.0
    metadataVersion: 3.9-IV0
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
      - name: tls
        port: 9093
        type: internal
        tls: true
    config:
      offsets.topic.replication.factor: 1
      transaction.state.log.replication.factor: 1
      transaction.state.log.min.isr: 1
      default.replication.factor: 1
      min.insync.replicas: 1
  entityOperator:
    topicOperator: {}
    userOperator: {}

Additional context

No response

[scholzj]

This is not how Kafka works. If you want to use load balancers, use the type: loadbalancer listener instead of creating your own service. You can read more about why and how for example in this blog post series: https://strimzi.io/blog/2019/04/17/accessing-kafka-part-1/