Skip to content

fix(hermes): fill the billing cache so server-tuned settings apply - #734

Draft
p-diogo wants to merge 3 commits into
mainfrom
fix/hermes-billing-cache
Draft

p-diogo wants to merge 3 commits into
mainfrom
fix/hermes-billing-cache

Conversation

@p-diogo

@p-diogo p-diogo commented Nov 4, 2026 •

Copy link
Copy Markdown
Collaborator

Draft: do not merge. Client merges are on hold until the audit sync. This PR also turns on server values that have never applied before, and two of them need a decision first (see "Decide before merge").

Refs #663 (billing cache never populated).

Bug

AgentState.set_billing_cache has no production caller, and on_session_start only reads state.get_cached_billing(), which is always None. So update_from_billing never runs, and none of these apply in Hermes today:

  • the relay's recall_top_k, extraction_interval, max_facts_per_extraction and max_candidate_pool
  • the >80% write-quota warning
  • the update-available notice (features.latest_stable_python)

The Python client also had no 403 cache invalidation at all.

Fix

  • _eager_account_register already calls client.status() once per configure. It now caches that response (BillingStatus → the dict shape update_from_billing reads), so the first session of a process makes no extra request.
  • Once the cached copy is older than the existing 2-hour TTL (BILLING_CACHE_TTL), on_session_start makes one refresh capped at 5 s (_BILLING_REFRESH_TIMEOUT_S). On a timeout or error it logs at DEBUG and keeps the current settings (the built-in defaults on a fresh process). It never raises and never blocks the session. If the eager register failed in the same session, no refresh is attempted, so a down relay costs one request per session, not two.
  • A write refused for quota (relay 403, or an error mentioning "quota", the same check the plugin uses) calls the new state.invalidate_billing_cache(). The next session then re-fetches and shows the warning instead of waiting out the TTL.
  • Env overrides still beat the server (TOTALRECLAW_RECALL_TOP_K, TOTALRECLAW_EXTRACT_INTERVAL, TOTALRECLAW_MAX_FACTS_PER_EXTRACTION, CANDIDATE_POOL_MAX_FREE / _PRO). That logic is unchanged.
  • get_recall_top_k's docstring listed the precedence in the wrong order. It now reads env > server > default, which is what the code does.

What switches on: server values vs today's defaults

Queried 2026-11-04 with a throwaway account (random auth key, not derived from a phrase), free tier, full features object:

setting Python default today staging serves production serves user-visible effect
recall_top_k 16 16 16 none (staging override removed 2026-11-04)
extraction_interval 3 3 3 none
max_facts_per_extraction 15 15 15 none
max_candidate_pool 250 2000 2000 none: capped at 250 in Hermes (MAX_SERVER_CANDIDATE_POOL, Pedro 2026-11-04). Lifting the cap waits on a large-vault measurement, internal tracker #696. Env overrides are not capped.
write-quota warning (free_writes_used / free_writes_limit > 0.8) never fires (dead) limit 50,000, so effectively never limit 250: fires above 200 writes New one-line warning at session start for free users near the cap.
update notice (latest_stable_python) never fires (dead) 2.5.3 2.5.3 Users on a stable older than 2.5.3 see "TotalReclaw 2.5.3 is available" at most once a day. RC builds and 2.5.3 see nothing. TOTALRECLAW_DISABLE_UPDATE_NOTICE=1 turns it off.
min_extract_interval (5), custom_extract_interval (false), llm_dedup (true), ingest_*, enclave_enabled not read by Python as listed same none. Note that min_extract_interval 5 > extraction_interval 3 on the relay; Python ignores the min, so nothing changes, but the relay config looks inconsistent.

reads_quota_mode is observe with no reads_limit in both environments, so the larger candidate pool cannot trip a read quota today.

Decide before merge

  1. max_candidate_pool 250 → 2000 for Hermes. Decided 2026-11-04: capped at 250 (commit 59c38f2). The relay value applies only below 250; CANDIDATE_POOL_MAX_FREE / _PRO still override it uncapped. Measurement tracked internally (feat(hermes): restore built-in memory once after upgrade; log "no changes" consistently #696), after the next real ChatGPT/Gemini export test.

  2. Staging recall_top_k = 8. Resolved 2026-11-04: staging RECALL_TOP_K removed and redeployed; both relays serve 16.

Tests

New tests/test_hermes_billing_cache_fill.py (11 tests, all driving the real on_session_start with a stubbed client):

  • the cache is filled from the eager register's response, server values apply, and there is exactly one relay request;
  • a second session within the TTL makes no request, and an expired cache is refreshed once;
  • relay failure leaves the defaults, raises nothing, and makes no second request;
  • a slow relay times out (patched to 50 ms), session start returns within 2 s, and earlier settings stay;
  • env overrides beat server values for all four settings;
  • the quota warning fires at 210/250 and stays silent at 200/250;
  • 403 detection works, and an invalidated cache is re-fetched at the next session start.

8 of the 11 fail on origin/main and all pass with the fix.

Full suite: python -m pytest tests/ -q -n 8 (HOME pointed at a temp dir): 2764 passed, 15 skipped, 1 xfailed.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Bd3G1BkVMFSXkcdAQ7nFax

AgentState.set_billing_cache had no production caller, so
on_session_start always read an empty cache and update_from_billing
never ran: relay recall_top_k, extraction_interval,
max_facts_per_extraction, max_candidate_pool, the >80% quota warning
and the update notice were all dead.

- The eager account register caches the client.status() response it
  already fetches (no extra request on the first session).
- Once the cached copy passes its 2h TTL, on_session_start makes one
  refresh bounded by 5 s. Failure or timeout keeps current settings
  and never blocks the session; no refresh after a failed eager
  register in the same session.
- A write refused for quota (403) invalidates the cache so the next
  session re-fetches and warns.
- Env overrides still beat server values.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bd3G1BkVMFSXkcdAQ7nFax
p-diogo and others added 2 commits November 4, 2026 03:25
The relay serves max_candidate_pool=2000, which Hermes has never used.
Keep today's 250 until a large-vault measurement; env overrides are not
capped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bd3G1BkVMFSXkcdAQ7nFax

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant