Skip to content

feat(env): Add automatic memory limit handling#457

Open
dongjiang1989 wants to merge 5 commits intokubernetes-sigs:mainfrom
dongjiang1989:support-gomemlimit
Open

feat(env): Add automatic memory limit handling#457
dongjiang1989 wants to merge 5 commits intokubernetes-sigs:mainfrom
dongjiang1989:support-gomemlimit

Conversation

@dongjiang1989
Copy link
Copy Markdown
Member

add auto GOMEMLIMIT setting

Signed-off-by: dongjiang1989 <dongjiang1989@126.com>
@netlify
Copy link
Copy Markdown

netlify bot commented Mar 21, 2026

Deploy Preview for agent-sandbox canceled.

Name Link
🔨 Latest commit efe3477
🔍 Latest deploy log https://app.netlify.com/projects/agent-sandbox/deploys/69d23ce35c248d0008f38764

@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Mar 21, 2026
Signed-off-by: dongjiang1989 <dongjiang1989@126.com>
@barney-s
Copy link
Copy Markdown
Contributor

Seems like a nifty improvement. What are the downsides of automatically setting it to x% of available mem ? Is this a common pattern ?

@dongjiang1989
Copy link
Copy Markdown
Member Author

Seems like a nifty improvement. What are the downsides of automatically setting it to x% of available mem ? Is this a common pattern ?

90% of available mem. Good user case in prometheus-operator and kube-state-metrics

go 1.26.1

require (
github.com/KimMachineGun/automemlimit v0.7.5
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should be careful about taking external dependencies; they have a habit of become unmaintained and then we're stuck maintaining them, so it's often easier just to reimplement them. I think the question about the default behaviour of go is the most important one, though, more than this dependency (it is MIT licensed and looks to be maintained)

@justinsb justinsb self-assigned this Apr 2, 2026
@justinsb
Copy link
Copy Markdown
Contributor

justinsb commented Apr 2, 2026

So I actually thought we'd fixed this in go, but it looks like we haven't yet.

I know @BenTheElder has tackled this in other kubernetes projects - any advice for us?

Signed-off-by: dongjiang <dongjiang1989@126.com>
@k8s-ci-robot
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: dongjiang1989
Once this PR has been reviewed and has the lgtm label, please ask for approval from justinsb. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 3, 2026
@barney-s
Copy link
Copy Markdown
Contributor

barney-s commented Apr 3, 2026

@dongjiang1989

How does it compare to using downward API to get the memlimit and using that during startup (main.go) to set the

env:
- name: K8S_MEM_LIMIT
  valueFrom:
    resourceFieldRef:
      containerName: my-container
      resource: limits.memory
import "runtime/debug"

func main() {
    // Set a limit of 1GB (1024 * 1024 * 1024 bytes)
    // limit  = envFrom(K8S_MEM_LIMIT)
    debug.SetMemoryLimit(0.8*limit) 
}

@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 5, 2026
Signed-off-by: dongjiang <dongjiang1989@126.com>
@dongjiang1989
Copy link
Copy Markdown
Member Author

dongjiang1989 commented Apr 5, 2026

@dongjiang1989

How does it compare to using downward API to get the memlimit and using that during startup (main.go) to set the

env:
- name: K8S_MEM_LIMIT
  valueFrom:
    resourceFieldRef:
      containerName: my-container
      resource: limits.memory
import "runtime/debug"

func main() {
    // Set a limit of 1GB (1024 * 1024 * 1024 bytes)
    // limit  = envFrom(K8S_MEM_LIMIT)
    debug.SetMemoryLimit(0.8*limit) 
}

Thanks @barney-s

Great question! The key difference is automation, portability, and reliability compared to the Downward API + manual debug.SetMemoryLimit() approach.

  1. No Kubernetes-specific YAML required
    automemlimit reads memory limits directly from cgroup (v1/v2), which works on all Linux container environments (K8s, Docker, systemd, etc.) — you don’t need to inject K8S_MEM_LIMIT via Downward API or maintain extra env config in YAML.

  2. Zero code boilerplate
    No manual env parsing, unit conversion (Ki/Mi/Gi), or math (0.8*limit) in your Go code.

  3. Built-in safety & best practices

    • Automatic fallback and error handling (won’t crash if no limit is found)
    • Default 90% ratio (cloud-native best practice)
    • Structured logging for observability
    • Proper handling of cgroup edge cases
  4. Less configuration drift
    Using the Downward API creates coupling between YAML and Go code. automemlimit eliminates that — the same binary works correctly everywhere without environment-specific setup.

In short: automemlimit does exactly what your manual code does, but automatically, portably, and safely across all containerized environments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants