Skip to content

fix(pulse): freshness pill judges ledgers by their write cadence - #2293

Open
jacobo-ortiz wants to merge 1 commit into
danielmiessler:mainfrom
jacobo-ortiz:fix/pulse-freshness-ledger-cadence
Open

jacobo-ortiz wants to merge 1 commit into
danielmiessler:mainfrom
jacobo-ortiz:fix/pulse-freshness-ledger-cadence

Conversation

@jacobo-ortiz

Copy link
Copy Markdown

Problem

The Performance tab's freshness pill expands MEMORY/OBSERVABILITY/ (PULSE/modules/tab-freshness.ts, the directory entry with expand: true) and keeps only files ending in .md, .json, .yaml or .yml. The ledgers the tab actually renders are .jsonl, so they never enter the measurement: the pill grades whatever other small .json files happen to live in that directory.

The grading scale is the second half of the problem. tierFromDays works in whole days (fresh up to 7 days, aging up to 30), a scale built for curated documents, not for a job that runs every 15 minutes.

Reproduced on 2026-10-07: with session-costs.jsonl 14 hours old and the aggregation job not having run since the previous evening, GET /api/tab-freshness?tab=performance answered tier: "fresh", daysOld: 0, backed by five unrelated .json files in the same directory. A cost meter that has stopped can show a green pill for up to seven days.

Fix

  • .jsonl joins the expansion filter (for every expand: true entry; today only MEMORY/OBSERVABILITY/ holds .jsonl files).
  • A registry entry may declare expectedEveryMinutes for a machine-written source with a known cadence. Its age in minutes is fresh within 2x the cadence, aging within 10x, stale beyond, and the worse of that verdict and the existing day-scale verdict becomes the tab's tier. Entries without the field behave exactly as before; a path listed explicitly and again through an expansion counts once.
  • The Performance tab registers the job's heartbeat, PULSE/Performance/aggregator-state.json, at 15 minutes: the cost-aggregation job rewrites it on every run, while it appends to session-costs.jsonl only when a new session appears (so the ledger's own mtime cannot be the cadence signal without false alarms during quiet hours). usage-daily.jsonl is registered at 1440 minutes, matching the nightly usage-aggregator job, and has no effect while that opt-in job is not installed. session-costs.jsonl now shows up through the expansion on the day scale. The heartbeat says the scan completed, not that it recorded anything: a run that finishes with every transcript unreadable still rewrites the state file.
  • The payload keeps daysOld and tier and adds ageMinutes and expectedEveryMinutes, top-level and per file, where a cadence applies; label names the lagging source when the cadence decided the tier.

What the UI shows today: the compact pill prints the tier plus the day-scale age, and the tooltip lists sources by date, so a stopped job now reads STALE with today's date. The client is untouched in this PR (it ignores the new optional fields); a follow-up can make formatAge prefer ageMinutes so the pill says "14 h" instead of "today".

Test

PULSE/modules/tab-freshness.test.ts (bun:test, synthetic fixtures under a temp HOME, no sleeps): a 14-hour-old heartbeat beside a fresh .json grades stale (red before the fix: fresh); 30 / 150 / 151 minutes grade fresh / aging / stale; an entry without a cadence keeps the day scale byte for byte; the registry-level case drives the real HTTP handler. bun test modules/tab-freshness.test.ts from LIFEOS/PULSE: 8 pass. This is the first *.test.ts in the tree; happy to move it wherever you prefer tests to live.

Notes

Not in this PR, same blind spot: the cost headline served by /api/performance/cost carries no age of its own, so a tab can show a fresh pill over a number that is hours old once the ledger stops being rewritten. A generatedAt on that response would close it.

The Performance tab's freshness pill expanded its MEMORY/OBSERVABILITY
source with a filter that admitted only .md, .json, .yaml and .yml
children, so the .jsonl ledgers the tab displays never entered the
measurement, and the tier was judged on a whole-day scale (fresh up to
seven days) against the single most recent file in the directory. A
ledger that had stopped being written for hours therefore rendered as
fresh, dated today, as long as any unrelated file in the directory was
recent.

The expansion now includes .jsonl, and a registry entry may declare
expectedEveryMinutes for a machine-written source with a known cadence:
its age in minutes is fresh within twice the cadence, aging within ten
times and stale beyond, and the worse of that verdict and the day-scale
verdict becomes the tab's tier; entries without the field behave exactly
as before, and a path listed explicitly and again through an expansion
counts once. The Performance tab registers the cost-aggregation state
file, which the */15 job rewrites on every run while it appends to
session-costs.jsonl only when a new session appears, at 15 minutes, and
usage-daily.jsonl at its nightly cadence. The payload keeps daysOld and
tier and adds ageMinutes and expectedEveryMinutes, top-level and per
file, where a cadence applies, and the label names the lagging source
when the cadence decided the tier; the compact pill still prints the
tier with the day-scale age, so a lagging source reads STALE with
today's date until the client learns to prefer ageMinutes. A bun test
next to the module drives the stale-to-fresh transition on synthetic
fixtures through the exported computeFreshness and the HTTP handler.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant