Problem Statement / Feature Objective
Each shard node runs a local capacity estimator (CPU, memory, bandwidth) that feeds into the global capacity coordinator. The local estimator and global coordinator can diverge because the coordinator uses a simpler linear model while the local estimator accounts for non-linear overheads like garbage collection pauses and NUMA effects.
Technical Invariants & Bounds
- Local estimator update interval: 1s.
- Global coordinator sync interval: 5s.
- Divergence tolerance: +/-10% of reported capacity.
- Overcommit ratio: max 1.2x physical capacity.
- NUMA node count: up to 8 per machine.
Codebase Navigation Guide
src/pool/capacity/local-estimator.rs — local resource estimation.src/pool/capacity/global-coordinator.rs — global capacity aggregation.src/pool/capacity/model-linear.rs — linear model used by coordinator.src/pool/capacity/model-nonlinear.rs — non-linear model used locally.
Implementation Blueprint
- In
local-estimator.rs, send both raw measurements and the locally-computed estimate to the coordinator.
- In
global-coordinator.rs, apply a correction factor: capacity_global = capacity_local * (1 - abs_diff(estimate_local, estimate_linear)).
- If divergence exceeds 10% for 3 consecutive sync cycles, log a
CapacityModelDivergence warning and use the more conservative of the two estimates.
- In
model-nonlinear.rs, add GC-pause modeling: reduce available capacity by (gc_pause_ms / 1000) fraction for the next 10s.
- Integration test: run a node with simulated GC pressure (100ms pause every 5s), verify local and global estimates stay within 10% divergence.
Problem Statement / Feature Objective
Each shard node runs a local capacity estimator (CPU, memory, bandwidth) that feeds into the global capacity coordinator. The local estimator and global coordinator can diverge because the coordinator uses a simpler linear model while the local estimator accounts for non-linear overheads like garbage collection pauses and NUMA effects.
Technical Invariants & Bounds
Codebase Navigation Guide
src/pool/capacity/local-estimator.rs— local resource estimation.src/pool/capacity/global-coordinator.rs— global capacity aggregation.src/pool/capacity/model-linear.rs— linear model used by coordinator.src/pool/capacity/model-nonlinear.rs— non-linear model used locally.Implementation Blueprint
local-estimator.rs, send both raw measurements and the locally-computed estimate to the coordinator.global-coordinator.rs, apply a correction factor:capacity_global = capacity_local * (1 - abs_diff(estimate_local, estimate_linear)).CapacityModelDivergencewarning and use the more conservative of the two estimates.model-nonlinear.rs, add GC-pause modeling: reduce available capacity by(gc_pause_ms / 1000)fraction for the next 10s.