Metrics API

[simwr872]

Hello! Awesome project, I like it a lot and it has worked exactly as described. Here are some thoughts after using it for a while.

When a pod is scaled to zero, any entries in the metrics API are missing.

kubectl apply -f https://raw.githubusercontent.com/ctrox/zeropod/refs/heads/main/config/examples/nginx.yaml
kubectl get events --field-selector involvedObject.name=nginx-6d6b6c6f86-8zf9v --sort-by=.lastTimestamp | tail -n 2
# 2s          Normal    Running            pod/nginx-6d6b6c6f86-8zf9v   Restored container nginx in 127.250696ms
# 0s          Normal    ResizeCompleted    pod/nginx-6d6b6c6f86-8zf9v   Pod resize completed: {"containers":[{"name":"nginx","resources":{"requests":{"cpu":"100m","memory":"128Mi"}}}]}

# continuously curl and wait a bit...
kubectl top pod nginx-6d6b6c6f86-8zf9v
# NAME                     CPU(cores)   MEMORY(bytes)   
# nginx-6d6b6c6f86-8zf9v   1m           10Mi

# wait 10 seconds
kubectl get events   --field-selector involvedObject.name=nginx-6d6b6c6f86-8zf9v   --sort-by=.lastTimestamp | tail -n 2
# 3s          Normal    Scaled down        pod/nginx-6d6b6c6f86-8zf9v   Scaled down container nginx in 64.876188ms
# 1s          Normal    ResizeCompleted    pod/nginx-6d6b6c6f86-8zf9v   Pod resize completed: {"containers":[{"name":"nginx","resources":{"requests":{"cpu":"1m","memory":"1Ki"}}}]}
kubectl top pod nginx-6d6b6c6f86-8zf9v
# Error from server (NotFound): podmetrics.metrics.k8s.io "default/nginx-6d6b6c6f86-8zf9v" not found

I'm trying to combine zeropod with HPA autoscaling on cpu and memory. Please see the below documentation snippet regarding HPA missing metrics:

If there were any missing metrics, the control plane recomputes the average more conservatively, assuming those pods were consuming 100% of the desired value in case of a scale down, and 0% in case of a scale up.

The result is no scaling decision is made. You could work around this by tuning the scale windows but it would be convenient if the zeropod runtime could fake 0% cpu and memory usage when scaled down. Let me know your thoughts about this.

I'm not too familiar with eBPF but I was also wondering if the number of active TCP connections could be exposed as a metric.

[ctrox]

Thanks for reporting this! I converted it into an issue and I think I already found a fix. Regarding the active TCP connections metric, do you mind creating a separate issue for that? I would need to investigate a bit but what is possible for sure is a metric that shows the timestamp of the last TCP activity.

[simwr872]

Nice! I made a separate issue #115. It might be out of scope for zeropod to care about scaling beyond zero, but the issue/feature-request exists now.