[feat aga] Implement AGA endpoint resource references loading and monitoring #4458

shraddhabang · 2025-11-18T19:07:38Z

This PR implements two key components for the AWS Global Accelerator controller:

Endpoint Loaders: Dynamically loads and resolves endpoints from Kubernetes resources
Resource Monitoring: Watches referenced resources for changes to trigger reconciliation

Commit 1: [feat aga] Implement endpoint loader with DNS resolution

This commit implements the endpoint loading system for the AGA controller. It provides:

Dynamic loading of endpoints from Kubernetes resources (Services, Ingresses, Gateways)
DNS resolution to AWS load balancer ARNs
Efficient LRU caching with TTL-based invalidation
Detailed error reporting and status tracking
Comprehensive unit tests

The endpoint loader enables GlobalAccelerator resources to reference Kubernetes objects and automatically resolve them to the appropriate AWS resources.

Commit 2: [feat aga] Implement resource monitoring for referenced resources

This commit implements the resource monitoring system for the AGA controller. It provides:

Dynamic watching of referenced Kubernetes resources
Event handling to trigger reconciliation when resources change
Only watches resources that are actively referenced
Reference tracking between resources and GlobalAccelerators
Efficient watch management with cleanup for unreferenced resources
Unit tests for all monitoring components

This monitoring system ensures that when a referenced resource changes (e.g., a Service gets a new load balancer), the GlobalAccelerator is automatically reconciled to use the updated endpoint.

Note for temporary limitations for cross namespace reference

We want to allow references to Kubernetes resources (Services, Ingresses, Gateways) that exist in different namespaces from the GlobalAccelerator CR itself. This enables more flexible architectural patterns but requires careful security considerations. We will implement this later as we will need to come up with a proper cross-namespace reference system keeping security concerns in mind. For now in the current implementation cross-namespace references are detected but only result in a warning - this means:

The references won't work (the endpoint is marked as warning)
The GlobalAccelerator CR won't be rejected

Checklist

Added tests that cover your change (if possible)
Added/modified documentation as required (such as the README.md, or the docs directory)
Manually tested
Made sure the title of the PR is a good description that can go into the release notes

BONUS POINTS checklist: complete for good vibes and maybe prizes?! 🤯

Backfilled missing tests for code in same general area 🎉
Refactored something and made the world a better place 🌟

pkg/aga/dns_resolver.go

pkg/aga/resource_watcher.go

k8s-ci-robot · 2025-11-25T04:38:28Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: shraddhabang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [shraddhabang]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

controllers/aga/eventhandlers/resource_events.go

controllers/aga/globalaccelerator_controller.go

pkg/aga/dns_to_load_balancer_resolver.go

zac-nixon · 2025-11-25T05:33:00Z

pkg/aga/dns_to_load_balancer_resolver.go

+	return &DNSToLoadBalancerResolver{
+		elbv2Client: elbv2Client,
+		cache:       cache,
+		ttl:         5 * time.Minute, // Default TTL of 5 minutes


IMO, this is too short. DNS names / ARNs do not change that frequently. What do you think about increasing the cache time?

I kept it small because I wanted to remove the older dns from the cache and keep the cache upto date. Having large TTL will keep the older dns and invalid (Deleted) ones for longer time. If we encounter any older dns because the our k8s were not updated for some reason, we will resolve that to invalid arn. Hence I kept it for 5 minutes so that we get faster failure feedback. What do you think?

pkg/aga/dns_to_load_balancer_resolver_test.go

pkg/aga/endpoint_loader.go

zac-nixon · 2025-11-25T05:36:53Z

pkg/aga/endpoint_loader.go

+		return "", fmt.Errorf("object is not a Service")
+	}
+
+	if svc.Spec.Type != corev1.ServiceTypeLoadBalancer {


I believe NodePort types are supported too.
https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/guide/service/nlb/

pkg/aga/endpoint_loader.go

zac-nixon · 2025-11-25T05:39:10Z

pkg/aga/endpoint_loader.go

+		"fatal", fatal)
+
+	// Log individual endpoints
+	for i, endpoint := range endpoints {


nit: this will potentially construct a lot of strings. i would be mindful of leaving this in here.

Yes. I am aware of that. But I think this is crucial to log this info for troubleshooting since the endpoints work dynamically. And we need to know which endpoints we filtered out so that we can debug this better. We are not expecting many endpoints per accelerator so I think verbose level 1 logs are okay for this.

pkg/aga/endpoint_resources_manager.go

zac-nixon · 2025-11-25T05:42:28Z

pkg/aga/reference_tracker.go

+		}
+		t.resourceMap[resourceKey].Insert(gaKey.String())
+
+		t.logger.V(1).Info("Resource referenced by GA",


same comment about a potential lot of strings of constructed

Same reasoning as above.

pkg/shared_utils/aga_utils.go

zac-nixon · 2025-11-25T16:36:54Z

/lgtm
/approved

k8s-ci-robot requested review from shuqz and zac-nixon November 18, 2025 19:07

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. approved Indicates a PR has been approved by an approver from all required OWNERS files. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Nov 18, 2025

shraddhabang force-pushed the agaresourceloader branch from 54f87b6 to 9e99f31 Compare November 19, 2025 18:41

emdelaney reviewed Nov 19, 2025

View reviewed changes

pkg/aga/dns_resolver.go Outdated Show resolved Hide resolved

wweiwei-li reviewed Nov 20, 2025

View reviewed changes

pkg/aga/resource_watcher.go Show resolved Hide resolved

shraddhabang force-pushed the agaresourceloader branch 2 times, most recently from 89591fa to f2d0ac0 Compare November 21, 2025 18:20

shraddhabang changed the base branch from AGAController to main November 25, 2025 04:28

k8s-ci-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Nov 25, 2025

shraddhabang force-pushed the agaresourceloader branch from f2d0ac0 to 1287b0e Compare November 25, 2025 04:38

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 25, 2025