Skip to content

feat(resource): report node utilization on the resource endpoint - #50

Open
WenYuLuo wants to merge 1 commit into
inclusionAI:mainfrom
WenYuLuo:feat/resource-utilization
Open

feat(resource): report node utilization on the resource endpoint#50
WenYuLuo wants to merge 1 commit into
inclusionAI:mainfrom
WenYuLuo:feat/resource-utilization

Conversation

@WenYuLuo

@WenYuLuo WenYuLuo commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Summary

A node can still advertise scheduler capacity while its actual resource usage is close to exhaustion. This implements RFC #49 by adding cached utilization.cpu, memory, pid, fd, and disk fractions to the existing Unix HTTP /resource response.

The existing five-second loop collects host and visible cgroup observations, preserves valid zero values, and reports unavailable metrics as null. Disk occupancy comes from the same FilestoreDir filesystem snapshot as storage, before overcommit. Existing capacity fields and capability markers remain compatible.

The documentation defines data sources and coverage. FunctionSystem scheduling thresholds, specific pressure states, and isolation/recovery remain a separate consumer change.

Validation

Tested with Go 1.25.5 in Linux/arm64 containers:

  • make check-fmt — passed.
  • make vet — passed.
  • go test -p 4 -race ./pkg/resourcemanager ./pkg/volumemanager — passed.
  • TestModuleLiveUtilizationOverUnixSocket — passed with real procfs/cgroup discovery, real filestore statistics, the periodic loop, and HTTP over a Unix socket. The independent scheduler-capacity provider is stubbed in this test.
  • Deterministic tests cover cgroup v1/v2, same-scope ancestor usage/limit pairing, CPU resets and quota changes, unavailable/zero values, FD accounting, cached HTTP responses, independent provider failures, and disk overcommit independence.

GitHub CI subsequently passed Unit tests, Go vet, Storage integration, Generated protobuf bindings, and Generated BPF objects at commit 38d41ca2b3cbdaaeed11f9ffc441fe3fa4cbb56a.

make test was run but is not green in this local environment. Unmodified main at a8c6ab83b064729f7f1b23079daf002fefe1c88d reproduces the same failures in TestIsMountpoint, TestCalcluteCacheSize, and TestPrepareKataDirectoryRootfsSeedsMountTargets; the baseline also exhibited TestInterfaceShrinkRollbackOnFailure. A privileged retry with TESTFLAGS='-gcflags=all=-l -p 4' passes Kata, while the existing gomonkey-based utility/network tests still fail on this runner, including a fault in internal/util. These tests and their implementations are unchanged by this PR.

Refs #49.

Expose cached CPU, memory, task, FD, and filestore occupancy ratios
for advisory scheduling protection. Reuse the resource refresh loop
and preserve existing capacity and storage overcommit semantics.

Refs: inclusionAI#49
Signed-off-by: robbluo <luo1442@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant