fix: replace usage watchers with four-hour scans and manual refresh - #596
Conversation
The usage indexer watched each transcript root six levels deep. chokidar 5 has no fsevents dependency, so that registers one fs.watch handle per directory — and below `~/.claude/projects/<project>/` every directory is per-session scratch (`<uuid>/`, `<uuid>/tool-results/`) holding no transcript at all. On a machine with 63 projects that walk expanded to 5,194 handles and exhausted the process handle table about ten seconds after launch. The EMFILE that follows is not confined to this watcher: it starves every descriptor opened afterwards — SQLite, node-pty, the daemon socket — so the app dies on the next user interaction instead of reporting a watcher fault. No macOS crash report is produced, because the process takes an uncaught-exception exit rather than a signal, which makes the failure look unrelated to file watching. The error handler made it worse. It only logged, so chokidar's one error per failed directory registration ran unbounded: 16,357 / 16,548 / 17,576 EMFILE lines in three consecutive daily logs, with the table pinned the whole time. Watch one level deep instead, which reaches the same transcripts, and guard the error path the way gitFileWatcher already guards its own after dcouple#309: on EMFILE/ENFILE/ENOSPC close the watcher once and fall back to a periodic full re-scan. Depth 1 measured 2,437 handles and zero errors against 4,632-and-climbing at depth 6, which never reached `ready`. Depth 1 lowers the handle cost without bounding it — it still scales with transcript count — so the fallback is load-bearing, not decorative. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Q7EXeStASTdUbTQDoen9MR
|
Independent QA evidence for a6357c1, compared with exact base 1697f70. Verdict: product-bug-found; do not merge this head. These are the actual reproducible harnesses used in the fresh QA pass after implementation review. They load unmodified head/base source using
Existing-root live checks wait for watcher readiness, 500 ms settling, and 7.5 seconds after writes (beyond 1.5-second write stability and 3-second debounce). Missing-root checks wait 700 ms after startup before creating roots, then 7.5 seconds. Polling cadence uses a virtual interval clock: at 299999 ms nothing changes; at 300000 ms the real recursive scan persists the missing records. A busy scan can cause a tick to be skipped, so cadence is not a strict freshness limit. The error-race probe holds promise results after two real native stats and injects their failures in sequence through real chokidar registration. It observes unhandled rejections to permit cleanup. ENFILE/ENOSPC are injected, not system-wide exhaustion. The separate hard-limit subprocess test is real kernel EMFILE and did not reproduce that race. To reproduce, use a disposable checkout at the reviewed SHA with Node 22, installed pnpm dependencies and SQLite rebuilt for Node ( pnpm exec node tmp/pr-596-review/qa/harness.cjs
pnpm exec node tmp/pr-596-review/qa/lifecycle.cjs
pnpm exec node tmp/pr-596-review/qa/missing-native.cjs
pnpm exec node tmp/pr-596-review/qa/fd-runner.cjsScripts intentionally keep failed behavioral assertions and continue to cleanup. A zero process exit is not a passing product verdict: inspect Setup attempts were retained locally and not counted as product evidence: late stat interception timed out before it was installed early; soft-only fd limiting was ineffective; the first hard-limit run encountered lazy stdio setup under exhaustion. The final scripts initialize stdio first and the successful hard-limit output above supersedes those attempts. Pricing and application-level database/aggregator services are stubbed; the actual parser/scanner/repository and isolated SQLite execute. No forced close-promise rejection, UI, live quota display, external pricing or long-running handle-growth benchmark is claimed. harness.cjs — SHA-256 64b5661530f3e2266c4e86413c9239178889220366940ff0c44725efebe817ee// Reproduce with Node 22: pnpm exec node tmp/pr-596-review/qa/harness.cjs
const fs = require('node:fs');
const fsp = require('node:fs/promises');
const os = require('node:os');
const path = require('node:path');
const vm = require('node:vm');
const cp = require('node:child_process');
const {createRequire, syncBuiltinESMExports} = require('node:module');
const req = createRequire(path.resolve('main/package.json'));
const ts = req('typescript');
const nativeStat = fsp.stat;
let statInterceptor = null;
fsp.stat = function(p,...args) { return statInterceptor ? statInterceptor(p,...args) : nativeStat.call(this,p,...args); };
syncBuiltinESMExports();
const chokidar = req('chokidar');
const HEAD = 'a6357c1bff8e944fedc9d5b4c2314b0f6307cc8f';
const BASE = '1697f7096c4f628ad6c2a6ba790b6e54f433a17a';
const out = __dirname;
const sleep = ms => new Promise(r => setTimeout(r, ms));
async function until(fn, label, ms=10000) { const end=Date.now()+ms; while(!fn()) {if(Date.now()>end) throw Error('Timeout '+label); await sleep(20);} }
const results=[];
function check(name, ok, detail) { const row={name, result:ok?'PASS':'FAIL', detail}; results.push(row); console.log(JSON.stringify(row)); }
function source(sha,file) { return cp.execFileSync('git',['show',`${sha}:${file}`],{encoding:'utf8',maxBuffer:8e6}); }
function load(sha, home, overrides={}, globals={}) {
const cache=new Map();
const stubs={
os:{...os,homedir:()=>home},
'../database':{}, './usageAggregator':{},
'./openRouterPriceProvider':{OpenRouterPriceProvider:class {start(){} stop(){}}},
'../../utils/appDirectory':{getAppDirectory:()=>home}, ...overrides,
};
function module(file) {
if(cache.has(file)) return cache.get(file);
const exports={}; cache.set(file,exports);
const code=ts.transpileModule(source(sha,file),{compilerOptions:{module:ts.ModuleKind.CommonJS,target:ts.ScriptTarget.ES2022,esModuleInterop:true}}).outputText;
vm.runInNewContext(code,{exports,require:id=>{
if(id in stubs) return stubs[id];
if(id.startsWith('.')) return module(path.posix.normalize(path.posix.join(path.posix.dirname(file),id))+'.ts');
return req(id);
},console,Buffer,process,setImmediate,setTimeout,clearTimeout,setInterval,clearInterval,...globals},{filename:sha.slice(0,7)+':'+file});
return exports;
}
return {module, ...module('main/src/services/usage/usageManager.ts')};
}
function fixture(home,relative,body) {const p=path.join(home,relative);fs.mkdirSync(path.dirname(p),{recursive:true});fs.writeFileSync(p,body);return p;}
let seq=0;
function line(provider) {const n=++seq;return JSON.stringify(provider==='claude'?{type:'assistant',timestamp:new Date().toISOString(),sessionId:'synthetic-session',cwd:'/synthetic',message:{id:'qa-'+n,model:'claude-sonnet-4',usage:{input_tokens:10,output_tokens:2}}}:{timestamp:new Date(Date.now()+n).toISOString(),type:'event_msg',payload:{type:'token_count',info:{last_token_usage:{input_tokens:10,output_tokens:2,total_tokens:12},total_token_usage:{input_tokens:10*n,output_tokens:2*n,total_tokens:12*n}}}})+'\n';}
const layouts=[['claude','.claude/projects/project/session.jsonl'],['claude','.claude/projects/project/session/subagents/agent-a.jsonl'],['codex','.codex/sessions/2026/09/09/rollout-a.jsonl']];
function database(sha,loaded) {const Db=req('better-sqlite3-multiple-ciphers');const db=new Db(':memory:');const test=source(sha,'main/src/services/usage/usageRepository.test.ts');db.exec(test.match(/db\.exec\(`([\s\S]*?)`\)/)[1]);loaded.usageManager.repositoryRef=new (loaded.module('main/src/services/usage/usageRepository.ts').UsageRepository)(db);return db;}
function clock() {let now=0;const timers=new Map();return {timers,globals:{setInterval(fn,ms){const h={unref(){h.unreferenced=true;}};timers.set(h,{fn,ms,next:now+ms});return h;},clearInterval(h){timers.delete(h);}},advance(ms){now+=ms;for(const t of timers.values())if(t.next<=now){t.next+=t.ms;t.fn();}}};}
async function integration(sha,root) {
const home=path.join(root,sha.slice(0,7));const c=clock();const loaded=load(sha,home,{},c.globals);const m=loaded.usageManager;const db=database(sha,loaded);
const files=layouts.map(([p,r])=>fixture(home,r,line(p)));
const count=p=>db.prepare('SELECT count(*) n FROM usage_events WHERE source_path=?').get(p).n;
try {
await m.start();const watchers=[...m.watchers];await Promise.all(watchers.map(w=>new Promise(r=>w.once('ready',r))));await until(()=>!m.scanning,'initial scan');
check(sha+' initial scan persists every layout',files.every(p=>count(p)===1),files.map(p=>[path.relative(home,p),count(p)]));
await sleep(500);
const fresh=files.map((p,i)=>{fs.appendFileSync(p,line(layouts[i][0]));const next=path.join(path.dirname(p),'fresh.jsonl');fs.writeFileSync(next,line(layouts[i][0]));return next;});
const newDirs=layouts.map(([p,r],i)=>fixture(home,i===0?'.claude/projects/new-project/fresh.jsonl':i===1?'.claude/projects/new-project/new-session/subagents/fresh.jsonl':'.codex/sessions/2026/10/01/fresh.jsonl',line(p)));
await sleep(7500);
files.forEach((p,i)=>check(sha+' native append '+layouts[i][1],count(p)===2,{events:count(p),expected:2}));
fresh.forEach((p,i)=>check(sha+' native create '+layouts[i][1],count(p)===1,{events:count(p),expected:1}));
newDirs.forEach((p,i)=>check(sha+' new directory discovery '+layouts[i][1],count(p)===1,{events:count(p),expected:1}));
check(sha+' healthy mode has no periodic reconciliation',c.timers.size===0,{timers:c.timers.size});
if(sha===HEAD) {
// Real watcher error ingress; pending-operation error races are tested separately below in this disposable Node harness process.
watchers[0]._handleError(Object.assign(new Error('synthetic EMFILE'),{code:'EMFILE'}));
watchers[1]._handleError(Object.assign(new Error('synthetic ENFILE'),{code:'ENFILE'}));
check('fallback closes exhausted real watchers and installs one unref interval',watchers.every(w=>w.closed)&&c.timers.size===1&&[...c.timers.keys()][0].unreferenced,{closed:watchers.map(w=>w.closed),timers:c.timers.size,intervalMs:[...c.timers.values()][0].ms});
const before=db.prepare('SELECT count(*) n FROM usage_events').get().n;
c.advance(299999);await sleep(50);
check('fallback waits until five minutes',db.prepare('SELECT count(*) n FROM usage_events').get().n===before,{before,after:db.prepare('SELECT count(*) n FROM usage_events').get().n});
c.advance(1);await until(()=>!m.scanning,'fallback scan');
check('fallback catches every missed append/create/new directory in SQLite',files.every(p=>count(p)===2)&&[...fresh,...newDirs].every(p=>count(p)===1),{counts:[...files,...fresh,...newDirs].map(count)});
check('degraded status remains visible after successful polling',m.status.lastError!==null,{watchMode:m.watchMode,lastError:m.status.lastError});
const later=fixture(home,'.codex/sessions/2027/01/01/later.jsonl',line('codex'));
c.advance(300000);await until(()=>!m.scanning,'second tick');check('second cadence discovers new year/month/day',count(later)===1,{events:count(later)});
}
await m.rescan();const rows=db.prepare('SELECT count(*) n FROM usage_events').get().n;await m.rescan();check(sha+' repeated scans preserve event idempotency',db.prepare('SELECT count(*) n FROM usage_events').get().n===rows,{events:rows});
m.stop();await Promise.all(watchers.map(w=>w.close()));check(sha+' stop disposes watchers and polling timer',m.watchers.length===0&&c.timers.size===0,{watchers:m.watchers.length,timers:c.timers.size});
const stopped=fixture(home,'.claude/projects/project/while-stopped.jsonl',line('claude'));await sleep(100);check(sha+' stopped manager does not index new file',count(stopped)===0,{events:count(stopped)});
await m.start();await until(()=>!m.scanning,'restart scan');check(sha+' restart discovers stopped-time writes',count(stopped)===1,{events:count(stopped),watchers:m.watchers.length});
const n=m.watchers.length;await m.start();check(sha+' repeated start does not duplicate watchers',m.watchers.length===n,{watchers:n});
} finally {const ws=[...m.watchers];m.stop();await Promise.all(ws.map(w=>w.close()));db.close();}
}
async function lifecycle(sha,root) {
const home=path.join(root,'lifecycle-'+sha.slice(0,7));const c=clock();let release;let entered=0;let active=0,max=0;const gate=new Promise(r=>release=r);
const l=load(sha,home,{},c.globals);const m=l.usageManager;const db=database(sha,l);const p=fixture(home,'.claude/projects/p/test.jsonl',line('claude'));
const scan=m.scanOne.bind(m);m.scanOne=async(...args)=>{entered++;active++;max=Math.max(max,active);await gate;await scan(...args);active--;};
try {
const first=m.runFullScan();await until(()=>entered===1,'blocked scan entered');const duplicate=m.runFullScan();await duplicate;
check(sha+' full scans serialize',entered===1,{entered});
m.pendingFiles.set(p,'claude');m.scheduleFlush();await until(()=>entered===2,'native debounce overlaps full scan',5000);
check(sha+' per-file indexing serialized across full scan and debounce',max===1,{maxConcurrent:max});
m.stop();release();await first;await until(()=>active===0,'post-stop scan drained');
check(sha+' stop prevents in-flight persistence',db.prepare('SELECT count(*) n FROM usage_events').get().n===0,{eventsAfterStop:db.prepare('SELECT count(*) n FROM usage_events').get().n});
// Stop before a pending debounce fires: no timer work, but queued path survives restart.
let calls=0;m.scanOne=async()=>{calls++;};m.pendingFiles.set(p,'claude');m.scheduleFlush();m.stop();await sleep(3200);
check(sha+' stop cancels pending debounce timer',calls===0,{calls,pendingPaths:m.pendingFiles.size});
}finally {release();m.stop();db.close();}
}
async function pending(sha,code,root) {
const home=path.join(root,'pending-'+code);fs.mkdirSync(home,{recursive:true});const a=fixture(home,'a.jsonl','{}\n'),b=fixture(home,'b.jsonl','{}\n');
const original=nativeStat;let armed=true;const pending=[];const rejections=[];const observer=e=>rejections.push({code:e.code,message:e.message});process.on('unhandledRejection',observer);
statInterceptor=async function(p,...args) {const native=await original.call(this,p,...args);if(armed&&(p===a||p===b))return new Promise((resolve,reject)=>pending.push({resolve,reject,native}));return native;};syncBuiltinESMExports();
let w;let degraded=0;
try {const l=load(sha,home);[w]=l.createTranscriptWatchers([{provider:'claude',path:home}],(_p,o)=>chokidar.watch([a,b],o),()=>{},()=>degraded++);
await until(()=>pending.length===2,'two real stats completed, registration pending');
pending[0].reject(Object.assign(new Error('synthetic '+code+' after native stat A'),{code}));await sleep(30);
pending[1].reject(Object.assign(new Error('synthetic '+code+' after native stat B'),{code}));await sleep(100);
check(sha+' pending native registration errors contained '+code,rejections.length===0,{degraded,closed:w.closed,errorListeners:w.listenerCount('error'),rejections,nativeStatsCompleted:pending.length});
} finally {armed=false;statInterceptor=null;if(w)await w.close();process.removeListener('unhandledRejection',observer);}
}
async function constrained(sha,root) {
// Invoked only in a child whose soft fd limit is 128. All opened data is synthetic.
process.stdout.write('');process.stderr.write('');
const home=path.join(root,'fd-'+sha.slice(0,7));const file=fixture(home,'p/transcript.jsonl','{}\n');const l=load(sha,home);const fds=[];const failures=[];let degraded=0;let w;let allocationError;
const observer=e=>failures.push({code:e.code,message:e.message});process.on('unhandledRejection',observer);
try {
for(let i=0;i<256;i++){try{fds.push(fs.openSync(file,'r'));}catch(e){allocationError=e.code;break;}}
[w]=l.createTranscriptWatchers([{provider:'claude',path:home}],(p,o)=>chokidar.watch(p,o),()=>{},()=>degraded++);
await sleep(1000);
fs.writeSync(1,JSON.stringify({check:'constrained-fd',sha,opened:fds.length,allocationError,degraded,closed:w.closed,errorListeners:w.listenerCount('error'),unhandled:failures})+'\n');
}finally {for(const fd of fds)fs.closeSync(fd);if(w)await w.close();process.removeListener('unhandledRejection',observer);}
}
async function main() {
if(process.argv[2]==='fd'){await constrained(process.argv[3],process.argv[4]);return;}
if(cp.execFileSync('git',['rev-parse','HEAD'],{encoding:'utf8'}).trim()!==HEAD)throw Error('HEAD changed');
const root=fs.mkdtempSync(path.join(os.tmpdir(),'pane596-fresh-qa-'));console.log(JSON.stringify({started:new Date().toISOString(),head:HEAD,base:BASE,node:process.version,syntheticRoot:root}));
try {
for(const sha of [HEAD,BASE]) {await integration(sha,root);await lifecycle(sha,root);for(const code of ['EMFILE','ENFILE','ENOSPC'])await pending(sha,code,root);}
for(const sha of [HEAD,BASE]) {const child=cp.spawnSync('/bin/zsh',['-c','ulimit -n 128; exec "$1" "$2" fd "$3" "$4"','qa',process.execPath,__filename,sha,root],{encoding:'utf8',timeout:15000,maxBuffer:2e6});fs.writeFileSync(path.join(out,'fd-'+sha.slice(0,7)+'.log'),child.stdout+'\nSTDERR\n'+child.stderr);check(sha+' constrained-fd child completed',child.status===0,{status:child.status,signal:child.signal,error:child.error?.message,stdout:child.stdout});}
}finally {fs.rmSync(root,{recursive:true,force:true});console.log(JSON.stringify({cleanup:root,removed:!fs.existsSync(root)}));fs.writeFileSync(path.join(out,'assertions.json'),JSON.stringify(results,null,2));}
console.log(JSON.stringify({verdict:results.some(r=>r.result==='FAIL')?'product-bug-found':'all-proven',passed:results.filter(r=>r.result==='PASS').length,failed:results.filter(r=>r.result==='FAIL').length,finished:new Date().toISOString()}));
}
module.exports={load,database,fixture,line,clock,check,until,sleep,HEAD,BASE,results,source,pending};
if(require.main===module)main().catch(e=>{console.error(e);process.exitCode=1;});lifecycle.cjs — SHA-256 825e42f8d61a8d430d704b06ad95481be62eef330821175def23593d46f80958const fs=require('node:fs'),os=require('node:os'),path=require('node:path');
const {load,database,fixture,line,clock,check,until,sleep,HEAD,BASE,results}=require('./harness.cjs');
const root=fs.mkdtempSync(path.join(os.tmpdir(),'pane596-lifecycle-'));
async function cursor(sha) {
const home=path.join(root,'cursor-'+sha.slice(0,7));
const plain=load(sha,home);const real=plain.module('main/src/services/usage/jsonlScanner.ts');
let release;const gate=new Promise(r=>release=r);let firstDone=false,number=0;
const l=load(sha,home,{'./jsonlScanner':{...real,async scanJsonlFile(...args){const value=await real.scanJsonlFile(...args);if(++number===1){firstDone=true;await gate;}return value;}}});
const m=l.usageManager,db=database(sha,l),p=fixture(home,'.claude/projects/p/a.jsonl',line('claude'));
try {
const older=m.scanOne(p,'claude');await until(()=>firstDone,'first scan real read finished');
fs.appendFileSync(p,line('claude'));await m.scanOne(p,'claude');const newer=m.repositoryRef.getFileCursor(p).offsetBytes;
release();await older;const after=m.repositoryRef.getFileCursor(p).offsetBytes;
check(sha+' cursor never regresses when older read commits last',after>=newer,{newer,after});
await m.rescan();check(sha+' stale cursor recovers without duplicate events',m.repositoryRef.countEvents()===2&&m.repositoryRef.getFileCursor(p).offsetBytes===fs.statSync(p).size,{events:m.repositoryRef.countEvents(),offset:m.repositoryRef.getFileCursor(p).offsetBytes,size:fs.statSync(p).size});
}finally{release();m.stop();db.close();}
}
async function restartPending(sha) {
const home=path.join(root,'restart-'+sha.slice(0,7));const l=load(sha,home),m=l.usageManager,db=database(sha,l);
let release;const gate=new Promise(r=>release=r);let entered=false;const actual=m.scanOne.bind(m);m.scanOne=async(...args)=>{entered=true;await gate;return actual(...args);};
fixture(home,'.claude/projects/p/old.jsonl',line('claude'));
try {await m.start();await until(()=>entered,'startup pending');const old=[...m.watchers];m.stop();await Promise.all(old.map(w=>w.close()));
const added=fixture(home,'.codex/sessions/2026/09/10/new.jsonl',line('codex'));
await m.start();const next=[...m.watchers];await Promise.all(next.map(w=>new Promise(r=>w.once('ready',r))));release();await until(()=>!m.scanning,'old scan drained');await sleep(300);
check(sha+' restart reconciles files created while older scan pending',m.repositoryRef.getFileCursor(added)!==null,{indexed:!!m.repositoryRef.getFileCursor(added)});
await m.rescan();check(sha+' explicit rescan recovers pending-restart omission',!!m.repositoryRef.getFileCursor(added),{});
}finally{release();const w=[...m.watchers];m.stop();await Promise.all(w.map(w=>w.close()));db.close();}
}
async function missingFallback() {
const home=path.join(root,'missing-fallback');fs.mkdirSync(home,{recursive:true});const c=clock(),l=load(HEAD,home,{},c.globals),m=l.usageManager,db=database(HEAD,l);
try {await m.start();await until(()=>!m.scanning,'missing initial roots');check('initial scan reports both missing synthetic roots',m.status.missingRoots.length===2,{missing:m.status.missingRoots});
await sleep(200);for(const w of m.watchers)w._handleError(Object.assign(new Error('synthetic ENOSPC'),{code:'ENOSPC'}));
const paths=[fixture(home,'.claude/projects/p/s/subagents/a.jsonl',line('claude')),fixture(home,'.codex/sessions/2026/12/31/b.jsonl',line('codex'))];
c.advance(300000);await until(()=>!m.scanning,'new root polling');check('fallback discovers roots created after startup',paths.every(p=>!!m.repositoryRef.getFileCursor(p))&&m.status.missingRoots.length===0,{events:m.repositoryRef.countEvents(),missing:m.status.missingRoots});
// Hold an actual full-scan file read across a scheduled tick. New file is absent from its already captured glob.
let release;const gate=new Promise(r=>release=r);let entered=false;const actual=m.scanOne.bind(m);m.scanOne=async(...args)=>{entered=true;await gate;return actual(...args);};
const busy=m.runFullScan();await until(()=>entered,'polling scan held');const later=fixture(home,'.codex/sessions/2027/01/01/later.jsonl',line('codex'));
c.advance(300000);release();await busy;
check('five minutes is a strict freshness bound during busy scan',!!m.repositoryRef.getFileCursor(later),{indexedAfterSkippedTick:!!m.repositoryRef.getFileCursor(later)});
c.advance(300000);await until(()=>!m.scanning,'next nonbusy tick');check('next nonbusy polling tick catches deferred discovery',!!m.repositoryRef.getFileCursor(later),{});
m.stop();c.advance(600000);check('no polling timer callbacks survive stop',c.timers.size===0,{timers:c.timers.size});
}finally{const w=[...m.watchers];m.stop();await Promise.all(w.map(w=>w.close()));db.close();}
}
(async()=>{try{for(const sha of [HEAD,BASE]){await cursor(sha);await restartPending(sha);}await missingFallback();}finally{fs.rmSync(root,{recursive:true,force:true});console.log(JSON.stringify({cleanup:root,removed:!fs.existsSync(root)}));fs.writeFileSync(path.join(__dirname,'lifecycle-assertions.json'),JSON.stringify(results,null,2));}})().catch(e=>{console.error(e);process.exitCode=1;});missing-native.cjs — SHA-256 011aa688951191204858afda625f192eb32b9db43321af12a8bb06eb64ff2b04const fs=require('node:fs'),os=require('node:os'),path=require('node:path');
const {load,database,fixture,line,check,until,sleep,HEAD,BASE,results}=require('./harness.cjs');
const root=fs.mkdtempSync(path.join(os.tmpdir(),'pane596-missing-native-'));
async function probe(sha,parentsExist){
const home=path.join(root,sha.slice(0,7)+'-'+parentsExist);fs.mkdirSync(home,{recursive:true});if(parentsExist){fs.mkdirSync(path.join(home,'.claude'));fs.mkdirSync(path.join(home,'.codex'));}
const l=load(sha,home),m=l.usageManager,db=database(sha,l);
try {await m.start();await until(()=>!m.scanning,'initial missing roots');await sleep(700);
const layouts=[['claude','.claude/projects/project/session.jsonl'],['claude','.claude/projects/project/session/subagents/agent.jsonl'],['codex','.codex/sessions/2026/09/09/a.jsonl']];
const files=layouts.map(([provider,relative])=>fixture(home,relative,line(provider)));
await sleep(7500);
files.forEach((p,i)=>check(sha+' native discovers initially missing root parentsExist='+parentsExist+' '+layouts[i][1],!!m.repositoryRef.getFileCursor(p),{indexed:!!m.repositoryRef.getFileCursor(p)}));
await m.rescan();check(sha+' rescan discovers all later-created roots parentsExist='+parentsExist,files.every(p=>!!m.repositoryRef.getFileCursor(p)),{events:m.repositoryRef.countEvents()});
}finally{const ws=[...m.watchers];m.stop();await Promise.all(ws.map(w=>w.close()));db.close();}
}
(async()=>{try{await Promise.all([probe(HEAD,false),probe(BASE,false),probe(HEAD,true),probe(BASE,true)]);}finally{fs.rmSync(root,{recursive:true,force:true});console.log(JSON.stringify({cleanup:root,removed:!fs.existsSync(root)}));fs.writeFileSync(path.join(__dirname,'missing-native-assertions.json'),JSON.stringify(results,null,2));}})().catch(e=>{console.error(e);process.exitCode=1;});fd-runner.cjs — SHA-256 ef6bc7fe394efda61636bac9174c09ceb8eeb23c9a1d8036c26c96ec7eb6b0b2const fs=require('node:fs'),os=require('node:os'),path=require('node:path'),cp=require('node:child_process');
const root=fs.mkdtempSync(path.join(os.tmpdir(),'pane596-hard-fd-'));
try {
for(const sha of ['a6357c1bff8e944fedc9d5b4c2314b0f6307cc8f','1697f7096c4f628ad6c2a6ba790b6e54f433a17a']) {
const child=cp.spawnSync('/bin/zsh',['-c','ulimit -Sn 128; ulimit -Hn 128; exec "$1" "$2" fd "$3" "$4"','qa',process.execPath,path.join(__dirname,'harness.cjs'),sha,root],{encoding:'utf8',timeout:15000,maxBuffer:2e6});
fs.writeFileSync(path.join(__dirname,'hard-fd-'+sha.slice(0,7)+'.log'),child.stdout+'\nSTDERR\n'+child.stderr);
console.log(JSON.stringify({sha,status:child.status,signal:child.signal,error:child.error?.message,stdout:child.stdout,stderr:child.stderr}));
}
}finally{fs.rmSync(root,{recursive:true,force:true});console.log(JSON.stringify({cleanup:root,removed:!fs.existsSync(root)}));}Cleanup: all owned synthetic trees removed, SQLite databases closed, watchers/timers stopped and child processes exited. Only ignored local evidence files remain. Contributor source and real application data were unchanged. |
parsakhaz
left a comment
parsakhaz
left a comment
There was a problem hiding this comment.
Reviewed a6357c1bff8e944fedc9d5b4c2314b0f6307cc8f against 1697f7096c4f628ad6c2a6ba790b6e54f433a17a. Request changes: two introduced correctness regressions block merging this head. The inline findings cover lost live transcript indexing and unhandled errors during watcher teardown.
Independent implementation review and a fresh QA pass reproduced both issues using temporary synthetic transcripts, real chokidar 5, and head/base comparisons. QA also checked persisted SQLite usage records. No private transcript contents were read and no source changes were made.
Detailed QA evidence and reproducible scripts.
Verification on this head:
pnpm lintandpnpm typecheck: passed.- Usage suite: 116 passed. Full main suite: 935 passed, 2 skipped (Windows-only launcher test on macOS and opt-in benchmark; both unchanged from base).
- Code Quality, including functional smoke and macOS/Windows main tests: passed. React Doctor and both Socket checks: passed. The wrapper matrix was skipped because neither wrapper nor contract paths changed.
- CI originally stopped at the first-time-contributor approval gate (
action_required); approving the existing runs allowed them to execute. The initial local SQLite Electron/Node ABI mismatch was repaired with a Node 22 rebuild before successful reruns. Neither was accepted as a test pass.
The real resource-limit test is useful positive evidence: with a child-only hard 128-descriptor limit, opening synthetic files reached EMFILE and HEAD degraded/closed its watcher, while base only warned. That single-error run did not trigger the teardown race; the separate delayed-filesystem-operation probes reproduced it for EMFILE, ENFILE, and ENOSPC.
Fallback scans correctly catch up the tested layouts after the five-minute interval, discover later-created roots, and clean up their interval on normal stop. Five minutes is a cadence, not a strict freshness bound when another scan is busy. Cursor regression under concurrent indexing and scan persistence after stop also reproduce on base; they are documented existing limitations, not the new merge blockers. A lower-priority new issue is that a successful polling scan clears the degraded-mode warning while polling remains active.
The PR attributes the original failure to usage watching consuming the process descriptor budget. Shallower recursion omits watched paths and the single-error fallback releases its watcher; this review did not measure aggregate resource savings. The coverage loss and unsafe teardown still block merging. Please address the two inline findings and add behavioral regressions before re-reviewing a new head.
| const watcher = createWatcher(root.path, { | ||
| ignoreInitial: true, | ||
| depth: 6, | ||
| depth: TRANSCRIPT_WATCH_DEPTH, |
There was a problem hiding this comment.
[P1] Preserve native or polling coverage for all supported transcript layouts
This shared depth of 1 misses .codex/sessions/YYYY/MM/DD/*.jsonl (OpenAI layout) and .claude/projects/<project>/<session>/subagents/*.jsonl (Claude documentation). Initial recursive indexing still works, masking the regression: later deep appends/creates emit no events or errors, so the exhaustion-only polling fallback never activates.
Fresh synthetic QA with real chokidar and SQLite: base indexed appends and new files for top-level Claude (projects/<project>/*.jsonl), Claude subagents, and dated Codex files; HEAD updated only top-level Claude files. With .claude/.codex parents present but transcript roots created after startup, base indexed all three layouts and HEAD indexed none after 700 ms startup settling and 7.5 seconds after root/file creation (existing-root writes were checked after 7.5 seconds, beyond write stability plus debounce).
Preserve these paths while reducing scratch-directory handle use. If polling covers deeper transcripts, activate it without requiring an exhaustion error. Add real create/append and later-created-root tests; asserting the depth option alone does not prove coverage.
| if (code !== undefined && HANDLE_EXHAUSTION_CODES.has(code)) { | ||
| if (degraded) return; | ||
| degraded = true; | ||
| void watcher.close(); |
There was a problem hiding this comment.
[P1] Keep late filesystem failures contained after close removes the listener
Installed chokidar 5 close() synchronously calls removeAllListeners(), removing this error guard. A second in-flight stat rejection then reaches chokidar's _handleError(), which emits error without checking closed. With no listener, that produces an unhandled rejection instead of safely degrading.
Reproduced through real chokidar's asynchronous registration path: hold two filesystem stat results pending; reject the first with an exhaustion code, let close run, then reject the second. For each of EMFILE, ENFILE, and ENOSPC, HEAD degraded once and produced an unhandled rejection; the identical base-source run retained its listener and produced none. An observer captured the rejection to keep the synthetic test process alive; this was fault injection, not a claim that the separate kernel-EMFILE run crashed.
Contain errors throughout teardown and handle close failure paths. Add a real-chokidar or faithful-close regression. TestWatcher.close() currently only increments a counter, so its 500-error test preserves listeners that production removes and cannot prove the storm guard is safe.
parsakhaz
changed the title
2 participants
Summary
Usage transcript watchers can retain enough native handles to exhaust the process descriptor table, affecting terminals and the database. Replace them with serialized recursive scans at startup, every four hours, and on manual refresh. Related to #309.
Discovery covers Claude project transcripts, nested Claude subagents, and Codex year/month/day transcripts, including roots and parent directories created after startup. There are no native usage watchers or watcher-close error paths.
Behavior and UX
Scan cost
The steady schedule is six full discovery passes/day, plus startup and manual requests. The previously proposed five-minute schedule would run 288/day: this reduces scheduled passes by about 98%.
For D directories, F transcript files, and ΔB newly read bytes, filesystem discovery/metadata/parsing work is roughly O(D + F + ΔB) per pass, excluding database indexing/query costs. Unchanged transcripts are checked by size/mtime and are not reopened. Scratch directories are still traversed. Native watching previously targeted changed files after registration; polling deliberately trades that event-driven efficiency for eliminating persistent usage-watch handles. A longer interval lowers frequency, not the peak cost of a single deep scan.
Synthetic macOS / Node 22 measurement: 2,640 transcripts (62.5 MB), 4,800 scratch directories:
Peak concurrent transcript streams: 1. This is a synthetic measurement, not a bound for larger/slower filesystems.
Verification
Tested implementation: 5363fbf.
pnpm lintandpnpm typecheck: pass.pnpm build:mainandpnpm build:frontend: pass, including sandboxed preload verification.The earlier blockers reported on a6357c1 are addressed by the replacement implementation; earlier review and QA comments refer to that old head. New-head CI and renewed review remain separate from these local passes. Native watcher fault injection is no longer applicable to the removed path. Four-hour scheduling is tested with a controlled clock; no four-hour physical wait or full Electron end-to-end run is claimed.
All transcript fixtures are synthetic and were deleted. No private transcript corpus was used. Reproducible local evidence, scripts, screenshots and videos are retained under
tmp/pr-596-review/fix/final-qa/; scan-cost script:tmp/pr-596-review/fix/synthetic-cost.cjs. An inherited browser selector expectedpro_litewhile the UI rendered· pro_lite; the selector was corrected without changing plan-label rendering.Usage and Settings screenshots (synthetic data)
Narrow navigation and layout: