Skip to content

k8s_config : k8s cert-manager configuration error #18

@fridim

Description

@fridim

in k8s_config, i've got this error several times. It especially happens the first times k8s_config is run on a cluster.

TASK [k8s_config : k8s cert-manager configuration] ***********************************************************************************
changed: [localhost] => (item=Secret(v1) letsencrypt-route53-credentials-secret in cert-manager)
failed: [localhost] (item=ClusterIssuer(cert-manager.io/v1) letsencrypt in cert-manager) => {"_k8s_resources_idx": 1, "ansible_loop_var": "_k8s_resources_idx", "attempts": 1, "changed": false, "error": 500, "msg": "Failed to apply object: b'{\"kind\":\"Status\",\"apiVersion\":\"v1\",\"metadata\":{},\"status\":\"Failure\",\"message\":\"Internal error occurred: failed calling webhook \\\\\"webhook.cert-manager.io\\\\\": Post \\\\\"https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s\\\\\": no endpoints available for service \\\\\"cert-manager-webhook\\\\\"\",\"reason\":\"InternalError\",\"details\":{\"causes\":[{\"message\":\"failed calling webhook \\\\\"webhook.cert-manager.io\\\\\": Post \\\\\"https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s\\\\\": no endpoints available for service \\\\\"cert-manager-webhook\\\\\"\"}]},\"code\":500}\\n'", "reason": "Internal Server Error", "status": 500}

Before running k8s_config, I ensure all cluster-operators are up and running.

But then when i run k8s_config the first time, some cluster operators change state:

(k8s_config) [ec2-user@bastion ~]$ oc get co|awk '$3 == "False" || $4 == "True" || $5 == "True" {print}'
image-registry                             4.6.31    True        True          False      36h
kube-apiserver                             4.6.31    True        True          False      37h
openshift-apiserver                        4.6.31    True        False         True       37h

It looks like we miss a check + wait in the k8s_config playbooks.

I'm not attaching the full log to this issue, but can share it privately if needed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions