Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] eksctl unable to correctly determine EKS Cluster OIDC enablement status and reported warning #8147

Open
VardyNg opened this issue Jan 16, 2025 · 2 comments
Labels

Comments

@VardyNg
Copy link

VardyNg commented Jan 16, 2025

What were you trying to accomplish?

Deploy an EKS cluster using eksctl, configure add-on to use IRSA.

What happened?

when deploying eks cluster with iam.withOIDC: true, eksctl is unable to determine that the OIDC is enabled and reported warning message:

recommended policies were found for "vpc-cni" addon, but since OIDC is disabled on the cluster, eksctl cannot configure the requested permissions; the recommended way to provide IAM permissions for "vpc-cni" addon is via pod identity associations; after addon creation is completed, add all recommended policies to the config file, under `addon.PodIdentityAssociations`, and run `eksctl update addon`

However, eksctl was actually able to configure the add-on with necessary permission.

How to reproduce it?

Deploy using the following config:

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: eksctl-cluster-1
  region: us-east-1

nodeGroups:
  - name: ng-2
    instanceType: m5.large
    desiredCapacity: 2

addons:
  - name: aws-efs-csi-driver
    version: latest
  - name: vpc-cni
    version: latest

iam:
  withOIDC: true

With the above, the logs shows warning mentioning OIDC is disabled, despite it is indeed set to true in the config. However, after deploying, the cluster was able to config add-on with IRSA.

The following was a cluster created by the above config

kubectl get pods -A                                        c889f3c0ebbb: Tue Jan  7 16:27:45 2025

NAMESPACE     NAME                                 READY   STATUS    RESTARTS   AGE
kube-system   aws-node-jpmb2                       2/2     Running   0          89m
kube-system   aws-node-rzfjd                       2/2     Running   0          89m
kube-system   coredns-586b798467-2fpl6             1/1     Running   0          94m
kube-system   coredns-586b798467-mg6kw             1/1     Running   0          94m
kube-system   efs-csi-controller-7f5f784cc-5p4cm   3/3     Running   0          86m
kube-system   efs-csi-controller-7f5f784cc-glmzp   3/3     Running   0          86m
kube-system   efs-csi-node-6lrjq                   3/3     Running   0          86m
kube-system   efs-csi-node-rqn4r                   3/3     Running   0          86m
kube-system   kube-proxy-lsv6h                     1/1     Running   0          89m
kube-system   kube-proxy-mtx7j                     1/1     Running   0          89m

Logs

2025-01-07 14:43:15 [ℹ]  eksctl version 0.199.0
2025-01-07 14:43:15 [ℹ]  using region us-east-1
2025-01-07 14:43:16 [ℹ]  setting availability zones to [us-east-1a us-east-1c]
2025-01-07 14:43:16 [ℹ]  subnets for us-east-1a - public:192.168.0.0/19 private:192.168.64.0/19
2025-01-07 14:43:16 [ℹ]  subnets for us-east-1c - public:192.168.32.0/19 private:192.168.96.0/19
2025-01-07 14:43:17 [ℹ]  nodegroup "ng-2" will use "ami-0598665f9cea16ed5" [AmazonLinux2/1.30]
2025-01-07 14:43:17 [ℹ]  using Kubernetes version 1.30
2025-01-07 14:43:17 [ℹ]  creating EKS cluster "eksctl-cluster-1" in "us-east-1" region with un-managed nodes
2025-01-07 14:43:17 [ℹ]  1 nodegroup (ng-2) was included (based on the include/exclude rules)
2025-01-07 14:43:17 [ℹ]  will create a CloudFormation stack for cluster itself and 1 nodegroup stack(s)
2025-01-07 14:43:17 [ℹ]  if you encounter any issues, check CloudFormation console or try 'eksctl utils describe-stacks --region=us-east-1 --cluster=eksctl-cluster-1'
2025-01-07 14:43:17 [ℹ]  Kubernetes API endpoint access will use default of {publicAccess=true, privateAccess=false} for cluster "eksctl-cluster-1" in "us-east-1"
2025-01-07 14:43:17 [ℹ]  CloudWatch logging will not be enabled for cluster "eksctl-cluster-1" in "us-east-1"
2025-01-07 14:43:17 [ℹ]  you can enable it with 'eksctl utils update-cluster-logging --enable-types={SPECIFY-YOUR-LOG-TYPES-HERE (e.g. all)} --region=us-east-1 --cluster=eksctl-cluster-1'
2025-01-07 14:43:17 [ℹ]  default addons kube-proxy, coredns were not specified, will install them as EKS addons
2025-01-07 14:43:17 [ℹ]  
2 sequential tasks: { create cluster control plane "eksctl-cluster-1", 
    2 sequential sub-tasks: { 
        5 sequential sub-tasks: { 
            1 task: { create addons },
            wait for control plane to become ready,
            associate IAM OIDC provider,
            no tasks,
            update VPC CNI to use IRSA if required,
        },
        create nodegroup "ng-2",
    } 
}
2025-01-07 14:43:17 [ℹ]  building cluster stack "eksctl-eksctl-cluster-1-cluster"
2025-01-07 14:43:19 [ℹ]  deploying stack "eksctl-eksctl-cluster-1-cluster"
2025-01-07 14:43:49 [ℹ]  waiting for CloudFormation stack "eksctl-eksctl-cluster-1-cluster"
2025-01-07 14:44:20 [ℹ]  waiting for CloudFormation stack "eksctl-eksctl-cluster-1-cluster"
2025-01-07 14:45:21 [ℹ]  waiting for CloudFormation stack "eksctl-eksctl-cluster-1-cluster"
2025-01-07 14:46:23 [ℹ]  waiting for CloudFormation stack "eksctl-eksctl-cluster-1-cluster"
^[[C2025-01-07 14:47:24 [ℹ]  waiting for CloudFormation stack "eksctl-eksctl-cluster-1-cluster"
2025-01-07 14:48:25 [ℹ]  waiting for CloudFormation stack "eksctl-eksctl-cluster-1-cluster"
2025-01-07 14:49:26 [ℹ]  waiting for CloudFormation stack "eksctl-eksctl-cluster-1-cluster"
2025-01-07 14:50:26 [ℹ]  waiting for CloudFormation stack "eksctl-eksctl-cluster-1-cluster"
2025-01-07 14:51:28 [ℹ]  waiting for CloudFormation stack "eksctl-eksctl-cluster-1-cluster"
2025-01-07 14:52:28 [ℹ]  waiting for CloudFormation stack "eksctl-eksctl-cluster-1-cluster"
2025-01-07 14:53:29 [ℹ]  waiting for CloudFormation stack "eksctl-eksctl-cluster-1-cluster"
2025-01-07 14:53:35 [!]  recommended policies were found for "vpc-cni" addon, but since OIDC is disabled on the cluster, eksctl cannot configure the requested permissions; the recommended way to provide IAM permissions for "vpc-cni" addon is via pod identity associations; after addon creation is completed, add all recommended policies to the config file, under `addon.PodIdentityAssociations`, and run `eksctl update addon`
2025-01-07 14:53:35 [ℹ]  creating addon
2025-01-07 14:53:35 [ℹ]  successfully created addon
2025-01-07 14:53:36 [ℹ]  creating addon
2025-01-07 14:53:36 [ℹ]  successfully created addon
2025-01-07 14:53:36 [ℹ]  creating addon
2025-01-07 14:53:37 [ℹ]  successfully created addon
2025-01-07 14:55:41 [ℹ]  deploying stack "eksctl-eksctl-cluster-1-addon-vpc-cni"
2025-01-07 14:55:41 [ℹ]  waiting for CloudFormation stack "eksctl-eksctl-cluster-1-addon-vpc-cni"
2025-01-07 14:56:12 [ℹ]  waiting for CloudFormation stack "eksctl-eksctl-cluster-1-addon-vpc-cni"
2025-01-07 14:56:12 [ℹ]  updating addon
2025-01-07 14:56:22 [ℹ]  addon "vpc-cni" active
2025-01-07 14:56:22 [ℹ]  building nodegroup stack "eksctl-eksctl-cluster-1-nodegroup-ng-2"
2025-01-07 14:56:22 [ℹ]  --nodes-min=2 was set automatically for nodegroup ng-2
2025-01-07 14:56:22 [ℹ]  --nodes-max=2 was set automatically for nodegroup ng-2
2025-01-07 14:56:23 [ℹ]  deploying stack "eksctl-eksctl-cluster-1-nodegroup-ng-2"
2025-01-07 14:56:23 [ℹ]  waiting for CloudFormation stack "eksctl-eksctl-cluster-1-nodegroup-ng-2"
2025-01-07 14:56:54 [ℹ]  waiting for CloudFormation stack "eksctl-eksctl-cluster-1-nodegroup-ng-2"
2025-01-07 14:57:37 [ℹ]  waiting for CloudFormation stack "eksctl-eksctl-cluster-1-nodegroup-ng-2"
2025-01-07 14:58:48 [ℹ]  waiting for CloudFormation stack "eksctl-eksctl-cluster-1-nodegroup-ng-2"
2025-01-07 14:59:51 [ℹ]  waiting for CloudFormation stack "eksctl-eksctl-cluster-1-nodegroup-ng-2"
2025-01-07 14:59:51 [ℹ]  waiting for the control plane to become ready
2025-01-07 14:59:52 [✔]  saved kubeconfig as "/Users/vardyng/.kube/config"
2025-01-07 14:59:52 [ℹ]  no tasks
2025-01-07 14:59:52 [✔]  all EKS cluster resources for "eksctl-cluster-1" have been created
2025-01-07 14:59:52 [ℹ]  nodegroup "ng-2" has 2 node(s)
2025-01-07 14:59:52 [ℹ]  node "ip-192-168-31-223.ec2.internal" is ready
2025-01-07 14:59:52 [ℹ]  node "ip-192-168-51-130.ec2.internal" is ready
2025-01-07 14:59:52 [ℹ]  waiting for at least 2 node(s) to become ready in "ng-2"
2025-01-07 14:59:52 [ℹ]  nodegroup "ng-2" has 2 node(s)
2025-01-07 14:59:52 [ℹ]  node "ip-192-168-31-223.ec2.internal" is ready
2025-01-07 14:59:52 [ℹ]  node "ip-192-168-51-130.ec2.internal" is ready
2025-01-07 14:59:52 [✔]  created 1 nodegroup(s) in cluster "eksctl-cluster-1"
2025-01-07 14:59:53 [!]  the recommended way to provide IAM permissions for "aws-efs-csi-driver" addon is via pod identity associations; after addon creation is completed, run `eksctl utils migrate-to-pod-identity`
2025-01-07 14:59:53 [ℹ]  creating role using recommended policies for "aws-efs-csi-driver" addon
2025-01-07 14:59:53 [ℹ]  deploying stack "eksctl-eksctl-cluster-1-addon-aws-efs-csi-driver"
2025-01-07 14:59:53 [ℹ]  waiting for CloudFormation stack "eksctl-eksctl-cluster-1-addon-aws-efs-csi-driver"
2025-01-07 15:00:25 [ℹ]  waiting for CloudFormation stack "eksctl-eksctl-cluster-1-addon-aws-efs-csi-driver"
2025-01-07 15:01:06 [ℹ]  waiting for CloudFormation stack "eksctl-eksctl-cluster-1-addon-aws-efs-csi-driver"
2025-01-07 15:01:06 [ℹ]  creating addon
2025-01-07 15:02:10 [ℹ]  addon "aws-efs-csi-driver" active
2025-01-07 15:02:11 [ℹ]  kubectl command should work with "/Users/vardyng/.kube/config", try 'kubectl get nodes'
2025-01-07 15:02:11 [✔]  EKS cluster "eksctl-cluster-1" in "us-east-1" region is ready

Anything else we need to know?

macOS
donwloaded binary
named profile

Versions

$ eksctl info
eksctl version: 0.199.0
kubectl version: v1.32.0
OS: darwin
@VardyNg
Copy link
Author

VardyNg commented Jan 16, 2025

Internal ticket V1636526910

Copy link
Contributor

Hello VardyNg 👋 Thank you for opening an issue in eksctl project. The team will review the issue and aim to respond within 1-5 business days. Meanwhile, please read about the Contribution and Code of Conduct guidelines here. You can find out more information about eksctl on our website

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant