Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Update shoot management guide due to new cert-based kubeconfigs #267

Merged
merged 1 commit into from
Jan 10, 2024

Conversation

colder-is-better
Copy link
Contributor

After introducing Kubernetes 1.27, shoot clusters with Kubernetes 1.26 and beyond will use certificate-based kubeconfig files. We explain the difference between "old" static kubeconfigs and new certificate-based ones, and we show how to issue and use them regardless of the Kubernetes version our shoot cluster uses. Then, we explain how to rotate static or certificate-based kubeconfigs.

@colder-is-better colder-is-better marked this pull request as draft January 5, 2024 10:50
@colder-is-better
Copy link
Contributor Author

I am marking this as a draft since the kubeconfig CA rotation screenshots and prose still need to be finalized. Currently, I cannot access https://admin-dev.cleura.cloud (getting an "Internal server error" message). As soon as I can take my screenshots, I am amending and lifting the "draft" status.

Copy link
Contributor

@fghaas fghaas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of notes for now, and then I'll do a proper review when there's coverage for rotating certificate-based kubeconfigs.

@colder-is-better colder-is-better force-pushed the kubeconfig-mgm-up branch 3 times, most recently from 3e43b44 to 61de9b1 Compare January 5, 2024 15:18
@colder-is-better colder-is-better marked this pull request as ready for review January 5, 2024 15:19
Copy link
Contributor

@fghaas fghaas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a few suggestions, please see the inline comments.

@colder-is-better colder-is-better force-pushed the kubeconfig-mgm-up branch 2 times, most recently from 56d664e to 18c32c6 Compare January 8, 2024 09:36
Copy link
Contributor

@fghaas fghaas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a few minor comments still.

Comment on lines 101 to 108
- cluster:
certificate-authority-data: DATA+OMITTED
server: https://api.ghar.p268.staging-k8s.{{gui_domain}}
name: garden-p268--ghar-external
- cluster:
certificate-authority-data: DATA+OMITTED
server: https://api.ghar.p268.internal.staging-k8s.{{gui_domain}}
name: garden-p268--ghar-internal
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we want to use the k8s, not staging-k8s subdomain here.

And also we should explain what the difference between the "internal" and "external" cluster is.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I run kubectl config view against the dev-cloud shoot cluster, I get the staging-k8s subdomain. Are you suggesting I should manually change this into k8s?

Also, although I only have a suspicion about what those internal and external clusters refer to, when I run kubectl config view against the non dev-cloud shoot cluster, I don't see internal nor external.

In not so many words, I need more context and/or pointers before providing more details about those internal and external clusters.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, in that case let's merge this PR last, i.e. leave it open until we can verify the behavior of the production environment.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The three other PRs (#268, #269, #270) have landed, so please double-check this behaviour in production, and then update this PR (while simultaneously rebasing it on main).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After the upgrade has landed in production, no matter if we have a Kubernetes 1.26 or Kubernetes 1.27 shoot, issuing kubectl config view gives us something like the following:

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://api.gharnew.p43597.k8s.cleura.cloud
  name: garden-p43597--gharnew-external
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://api.gharnew.p43597.internal.k8s.cleura.cloud
  name: garden-p43597--gharnew-internal
contexts:
- context:
    cluster: garden-p43597--gharnew-external
    user: garden-p43597--gharnew-external
  name: garden-p43597--gharnew-external
- context:
    cluster: garden-p43597--gharnew-internal
    user: garden-p43597--gharnew-external
  name: garden-p43597--gharnew-internal
current-context: garden-p43597--gharnew-external
kind: Config
preferences: {}
users:
- name: garden-p43597--gharnew-external
  user:
    client-certificate-data: DATA+OMITTED
    client-key-data: DATA+OMITTED

Note that I did not replace actual data with DATA+OMITTED. That is in the actual output, along with the internal and external literals.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case, please update the PR accordingly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

@colder-is-better colder-is-better force-pushed the kubeconfig-mgm-up branch 4 times, most recently from 3cbf63e to e5827bd Compare January 10, 2024 11:31
After introducing Kubernetes 1.27, shoot clusters with Kubernetes
1.26 and beyond will use certificate-based kubeconfig files. We
explain the difference between "old" static kubeconfigs and new
certificate-based ones, and we show how to issue and use them
regardless of the Kubernetes version our shoot cluster uses. Then,
we explain how to rotate static or certificate-based kubeconfigs.
@fghaas fghaas merged commit 5288883 into citynetwork:main Jan 10, 2024
3 checks passed
@colder-is-better colder-is-better deleted the kubeconfig-mgm-up branch December 19, 2024 18:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants