fix: bound per-client rate limiter storage with LRU eviction

[strawgate]

RateLimitingMiddleware and SlidingWindowRateLimitingMiddleware stored per-client limiters in an unbounded defaultdict keyed by client id. Every distinct client id created a permanent entry that was never evicted, so a long-running server facing many transient clients (or attacker-supplied ids) leaked memory without bound.

This replaces the defaultdict with an OrderedDict used as an LRU cache, capped by a configurable max_clients (default 10,000). Accessing a client moves it to the most-recently-used end; inserting past the cap evicts the least-recently-used entry. Eviction is safe because an evicted client was idle long enough for max_clients other clients to be more recent — by which point its token bucket would have refilled (or its sliding window expired) anyway, so it is indistinguishable from a fresh client. max_clients < 1 is rejected at construction. The same fix is applied to both middlewares. Note the internal storage attribute is renamed limiters → _client_limiters to reflect that it is no longer a public auto-vivifying dict.

RateLimitingMiddleware(max_requests_per_second=10, max_clients=10_000)
# memory is now bounded; LRU clients are evicted instead of accumulating

Refs #4053 — fixes Bug 1 (rate-limiter dicts never evicted). Bugs 2 and 3 (FilesystemProvider lazy-lock race, unbounded _warned_files) are out of scope here.

[strawgate]

surely we can find a non-hand rolled ttl dict

[marvin-context-protocol]

Edited to reflect the latest analysis.

tl;dr: ruff format (via the static_analysis / prek hook) reformatted tests/server/middleware/test_rate_limiting.py. Run uv run ruff format . locally and push the result.

The ruff-format hook collapsed four RateLimitingMiddleware(get_client_id=lambda ctx: ctx, max_clients=N) calls in TestRateLimiterLRUEviction that were wrapped across three lines but fit on one. Every other hook (ruff check, ty, loq, codespell, prettier, validate-pyproject) passed; the only failure was the formatter rewriting that single file.

Log excerpt — job 76372877049

ruff format..............................................................Failed
- hook id: ruff-format
- files were modified by this hook

  1 file reformatted, 762 files left unchanged
...
diff --git a/tests/server/middleware/test_rate_limiting.py b/tests/server/middleware/test_rate_limiting.py
@@ -274,9 +274,7 @@ class TestRateLimiterLRUEviction:
     def test_token_bucket_evicts_lru_when_full(self):
-        mw = RateLimitingMiddleware(
-            get_client_id=lambda ctx: ctx, max_clients=2
-        )
+        mw = RateLimitingMiddleware(get_client_id=lambda ctx: ctx, max_clients=2)

(Three additional identical hunks at lines 282, 296, and 331 for max_clients=2/5/1.)

##[error]prek exited with code 1

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 30869a2e25

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".