feat: support dual-stack (IPv4 + IPv6) clusters for self-hosted nodes#1732
Open
MarcPow wants to merge 1 commit into
Open
feat: support dual-stack (IPv4 + IPv6) clusters for self-hosted nodes#1732MarcPow wants to merge 1 commit into
MarcPow wants to merge 1 commit into
Conversation
Provision dual-stack nodes on dual-stack (IPv4 + IPv6) Azure CNI clusters. Previously, nodes launched by Karpenter on a dual-stack cluster were IPv4-only: the network interface had only a primary IPv4 IP configuration, IPv6 load-balancer backend pools were never attached, and the bootstrap contract's IPv6 dual-stack flag was never set. This effectively made dual-stack clusters unsupported for the self-hosted (VM-direct) provisioning path. This change wires the cluster's IP families through to node provisioning so that, on a dual-stack cluster, each node receives both an IPv4 and an IPv6 address, mirroring how AKS configures dual-stack VMSS nodes. Validated live on a dual-stack Azure CNI Overlay cluster: a Karpenter- provisioned node's NIC came up with `ipconfig` (IPv4, primary) + `ipv6config` (IPv6, non-primary), the node registered both IPv4 and IPv6 InternalIPs, and workloads scheduled successfully. Details ------- - Add a `NodeIPFamilies` option (env `NODE_IP_FAMILIES`, default `IPv4`) and an `IsIPv6Enabled()` helper. The deploy script populates it from the cluster's `networkProfile.ipFamilies`, and it is threaded into the Helm values template. - LoadBalancer provider: classify backend pools by IP version and populate `BackendAddressPools.IPv6PoolIDs` from the well-known IPv6 inbound/outbound pools, instead of discarding them. - VM NIC (VM-direct path): on dual-stack clusters, append a single non-primary `ipv6config` IP configuration (dynamic, IPv6) with the IPv6 load-balancer backend pools attached. A NIC may only have one IPv6 ip configuration, so it is added once, outside the secondary-IP loop. Mirrors the AKS RP node NIC construction. - Bootstrap: set the existing (previously unused) `IPv6DualStackEnabled` node bootstrap variable from the cluster IP families, so the node's CSE bootstrap is configured for dual-stack. - Docs: scope the README "IPv6 clusters" limitation to single-stack IPv6 only; dual-stack (IPv4 + IPv6) is now supported. Scope and notes --------------- - This covers the self-hosted, VM-direct provisioning path (aksscriptless / bootstrappingclient). The AKS Machine API path (aksmachineapi / NAP) is unchanged: the AKS RP owns NIC and load-balancer construction there, and the Machine API has no per-machine IP-family field. Dual-stack support for NAP is gated off at the AKS RP and is tracked separately. - Single-stack IPv6-only clusters remain unsupported. Tests ----- - Unit tests for `IsIPv6Enabled`, LoadBalancer IPv6 pool classification, and the IPv6 NIC IP configuration on dual-stack vs single-stack clusters. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
df5e00a to
b4faac2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Provision dual-stack (IPv4 + IPv6) nodes on dual-stack Azure CNI clusters.
Previously, nodes launched by Karpenter on a dual-stack cluster were IPv4-only: the NIC had only a primary IPv4 IP configuration, IPv6 load-balancer backend pools were never attached, and the bootstrap contract's IPv6 dual-stack flag was never set. This effectively made dual-stack clusters unsupported for the self-hosted (VM-direct) provisioning path. This PR wires the cluster's IP families through to node provisioning so each node receives both an IPv4 and an IPv6 address, mirroring how AKS configures dual-stack VMSS nodes.
What this PR does
NodeIPFamiliesoption (envNODE_IP_FAMILIES, defaultIPv4) and anIsIPv6Enabled()helper. The deploy script (configure-values.sh) populates it from the cluster'snetworkProfile.ipFamilies, threaded through the Helm values template.BackendAddressPools.IPv6PoolIDsfrom the well-known IPv6 inbound/outbound pools (previously discarded).ipv6configIP configuration (dynamic, IPv6) with the IPv6 LB backend pools attached. A NIC may only have one IPv6 ip config, so it's added once, outside the secondary-IP loop. Mirrors the AKS RP node NIC construction.IPv6DualStackEnablednode bootstrap variable from the cluster IP families.Testing
IsIPv6Enabled; LoadBalancer IPv6 pool classification; the IPv6 NIC ip configuration on dual-stack vs single-stack clusters.ipFamilies: [ipv4, ipv6]):configure-values.shcorrectly auto-extractedNODE_IP_FAMILIES=ipv4,ipv6.ipconfig(IPv4, primary,10.224.0.6) andipv6config(IPv6, non-primary,fdd2:…::6) — matching the system VMSS reference NIC exactly.InternalIPaddresses, and workloads scheduled and ran.Scope & notes
aksscriptless/bootstrappingclient).aksmachineapi/ NAP) is unchanged: the AKS RP owns NIC and LB construction there, and the Machine API has no per-machine IP-family field. Dual-stack for NAP is currently gated off at the AKS RP and is tracked as separate RP work.