diff --git a/docs/perf-profiling-corpus.md b/docs/perf-profiling-corpus.md index 17b04b4f4a..c4a591eb0b 100644 --- a/docs/perf-profiling-corpus.md +++ b/docs/perf-profiling-corpus.md @@ -33,7 +33,7 @@ Optimization **status vocabulary** for a hotspot: A v1–v3 win is **never** called "confirmed" from current-only coverage. `validatePerfCorpusReport()` enforces this: a `CPU-self-time confirmed` classification is rejected unless the report carries profiler self-time evidence. -## Schema (gjc.perf-corpus/2) +## Schema (gjc.perf-corpus/3) `PerfCorpusReport` keeps the evidence classes as **separate named fields** per fixture: @@ -44,7 +44,7 @@ A v1–v3 win is **never** called "confirmed" from current-only coverage. `valid - `byteParity: { renderedGolden?, persistedJsonlGolden?, providerPayloadGolden?, materializedSessionGolden? }` - `memoryBaseline?: { surface, profile, iterations, operations, operationsPerSecond, samples, postTeardown, rssSlopeBytesPerSecond, heapSlopeBytesPerSecond, processTreeBaselineRssBytes, processTreePostTeardownRssBytes, processTreeSampler }` - `runner: { command, argv, environment, platform, arch, bunVersion?, ci?, profile, durationTargetMs?, memoryIsolation, iterationsTarget, gcExposed, memoryChildGcExposed, memoryChildExecArgv }` pins the actual parent argv, normalized workload controls, isolation, parent GC availability, and the fixed isolated-child runtime flags separately. -- `gitSha` is the full checked-out `HEAD` when Git is available, with `GITHUB_SHA` used only as a fallback; `gitDirty` explicitly marks tracked or untracked worktree changes so local evidence cannot silently masquerade as a clean commit. The runner captures SHA and the complete porcelain worktree fingerprint before and after the workloads and rejects any in-flight source-state change. +- `gitSha` is the full checked-out `HEAD`; unavailable Git provenance fails closed rather than trusting a workflow environment variable. `gitDirty` explicitly marks tracked or untracked worktree changes so local evidence cannot silently masquerade as a clean commit. The runner captures SHA and the complete porcelain worktree fingerprint before and after the workloads and rejects any in-flight source-state change. - Every detailed sample separates `rssBytes`, `heapUsedBytes`, `heapTotalBytes`, `externalBytes`, `arrayBuffersBytes`, and `activeResourceCount`. `hotspotClassifications: HotspotClassification[]` carry `{ hotspotId, status, evidenceClass, artifactRefs, notes }`. The current v1–v3 reclassification lives in `V1_V3_RECLASSIFICATION`; no entry is `CPU-self-time confirmed` because no profiler artifacts have been captured yet. @@ -98,7 +98,7 @@ Held thresholds (`HELD_PERF_THRESHOLDS`) name candidates that need variance char ## Memory baseline protocol Detailed memory fixtures cover seven explicit surfaces: CLI startup/configuration, AgentSession-style message/context lifecycle, blob/external buffers, worker generations, Telegram reconnect/queue settlement, TUI render/dispose churn, and shared/native transfer boundaries. The fixtures are synthetic lifecycle proxies: they establish a reproducible allocation and teardown envelope but do not by themselves prove a production leak. A production optimization claim still requires a workload adapter that exercises the implicated owner and a same-host before/after artifact. -The command-line runner executes each memory surface in a fresh Bun subprocess and records `runner.memoryIsolation: "process-per-surface"` so allocator high-water state from one fixture cannot contaminate the next surface's baseline. Programmatic `runPerfCorpusBenchmark()` defaults to in-process fixtures and records `"in-process"` for focused contract tests; pass `{ isolatedMemory: true }` for acceptance-equivalent evidence. Process-tree RSS snapshots exclude the `ps` sampler process and degrade both endpoints to `"unavailable"` when either snapshot fails. The process-tree baseline is captured after GC, followed by another GC that clears sampler allocations before the local baseline and workload begin. Soak workloads use single-iteration batches so approximately 50 ms sampling cannot be hidden behind a large synchronous chunk. Post-teardown return fields remain `null` when GC is unavailable. +The authenticated command-line runner is the only supported execution surface. It executes each memory surface in a fresh Bun subprocess and records `runner.memoryIsolation: "process-per-surface"` so allocator high-water state from one fixture cannot contaminate the next surface's baseline. The benchmark module intentionally exposes no programmatic runner because imported execution cannot satisfy the frozen process-argv contract. Process-tree RSS snapshots exclude the `ps` sampler process and degrade both endpoints to `"unavailable"` when either snapshot fails. The process-tree baseline is captured after GC, followed by another GC that clears sampler allocations before the local baseline and workload begin. Soak workloads use single-iteration batches so approximately 50 ms sampling cannot be hidden behind a large synchronous chunk. Post-teardown return fields remain `null` when GC is unavailable. Use the `short` profile for deterministic contract and shape checks; its bounded iteration window intentionally reports `null` slopes when less than 250 ms is observed. Use `soak` for repeated sampling and slope characterization. For decision evidence: The soak default runs each surface for at least one second and samples at approximately 50 ms intervals. `GJC_MEMORY_DURATION_MS` accepts 250–60000 ms and `GJC_MEMORY_ITERATIONS` accepts 1–10000000; record overrides with the artifact. diff --git a/packages/ai/CHANGELOG.md b/packages/ai/CHANGELOG.md index 6f4d842c10..e9e294c2e7 100644 --- a/packages/ai/CHANGELOG.md +++ b/packages/ai/CHANGELOG.md @@ -17,6 +17,8 @@ - Anthropic cache-control resolution now falls back to `model.cacheRetention` at the provider boundary, preserving configured retention and request-over-model precedence through special dispatch wrappers such as GitLab Duo. A configured `cacheRetention: "none"` can no longer be dropped and replaced by the new automatic Claude-family cache marker. - Anthropic explicit prompt caching now advances its conversation breakpoint during tool-use loops by marking the latest completed assistant tool-use turn while leaving the newest tool result uncached. Previously it kept refreshing only the original human message until another human turn arrived, pinning proxy cache reads to the static tools/system prefix throughout long agentic runs. +- SQLite-backed authentication storage now finalizes temporary and cached statements and closes its owned database connection, allowing settings directories and WAL files to be removed immediately on Windows. + ## [0.12.12] - 2026-08-05 ### Fixed diff --git a/packages/ai/src/auth-storage.ts b/packages/ai/src/auth-storage.ts index 1926869831..3f3ab6f5fe 100644 --- a/packages/ai/src/auth-storage.ts +++ b/packages/ai/src/auth-storage.ts @@ -4541,25 +4541,44 @@ export class SqliteAuthCredentialStore implements AuthCredentialStore { } #authCredentialsTableExists(): boolean { - const row = this.#db - .prepare("SELECT 1 AS present FROM sqlite_master WHERE type = 'table' AND name = 'auth_credentials'") - .get() as { present?: number } | undefined; - return row?.present === 1; + const stmt = this.#db.prepare( + "SELECT 1 AS present FROM sqlite_master WHERE type = 'table' AND name = 'auth_credentials'", + ); + try { + const row = stmt.get() as { present?: number } | undefined; + return row?.present === 1; + } finally { + stmt.finalize(); + } } #readAuthSchemaVersion(): number | null { - const row = this.#db.prepare("SELECT version FROM auth_schema_version WHERE id = 1").get() as - | { version?: number } - | undefined; - return typeof row?.version === "number" ? row.version : null; + const stmt = this.#db.prepare("SELECT version FROM auth_schema_version WHERE id = 1"); + try { + const row = stmt.get() as { version?: number } | undefined; + return typeof row?.version === "number" ? row.version : null; + } finally { + stmt.finalize(); + } } #writeAuthSchemaVersion(version: number): void { - this.#db.prepare("INSERT OR REPLACE INTO auth_schema_version(id, version) VALUES (1, ?)").run(version); + const stmt = this.#db.prepare("INSERT OR REPLACE INTO auth_schema_version(id, version) VALUES (1, ?)"); + try { + stmt.run(version); + } finally { + stmt.finalize(); + } } #inferAuthSchemaVersion(): number { - const cols = this.#db.prepare("PRAGMA table_info(auth_credentials)").all() as Array<{ name?: string }>; + const stmt = this.#db.prepare("PRAGMA table_info(auth_credentials)"); + let cols: Array<{ name?: string }>; + try { + cols = stmt.all() as Array<{ name?: string }>; + } finally { + stmt.finalize(); + } const hasDisabledCause = cols.some(column => column.name === "disabled_cause"); const hasIdentityKey = cols.some(column => column.name === "identity_key"); const hasAccountId = cols.some(column => column.name === "account_id"); @@ -4607,7 +4626,13 @@ export class SqliteAuthCredentialStore implements AuthCredentialStore { #migrateAuthSchemaV0ToV1(): void { const migrate = this.#db.transaction(() => { - const v0Cols = this.#db.prepare("PRAGMA table_info(auth_credentials)").all() as Array<{ name?: string }>; + const v0ColsStmt = this.#db.prepare("PRAGMA table_info(auth_credentials)"); + let v0Cols: Array<{ name?: string }>; + try { + v0Cols = v0ColsStmt.all() as Array<{ name?: string }>; + } finally { + v0ColsStmt.finalize(); + } const hasDisabled = v0Cols.some(col => col.name === "disabled"); this.#db.run("ALTER TABLE auth_credentials RENAME TO auth_credentials_v0"); @@ -4684,17 +4709,25 @@ export class SqliteAuthCredentialStore implements AuthCredentialStore { } #backfillCredentialIdentityKeys(): void { - const rows = this.#db - .prepare( - "SELECT id, provider, credential_type, data, disabled_cause, identity_key FROM auth_credentials WHERE identity_key IS NULL ORDER BY id ASC", - ) - .all() as AuthRow[]; + const selectStmt = this.#db.prepare( + "SELECT id, provider, credential_type, data, disabled_cause, identity_key FROM auth_credentials WHERE identity_key IS NULL ORDER BY id ASC", + ); + let rows: AuthRow[]; + try { + rows = selectStmt.all() as AuthRow[]; + } finally { + selectStmt.finalize(); + } if (rows.length === 0) return; - const updateIdentity = this.#db.prepare("UPDATE auth_credentials SET identity_key = ? WHERE id = ?"); - for (const row of rows) { - const identityKey = resolveRowCredentialIdentityKey(row.provider, row); - updateIdentity.run(identityKey, row.id); + const updateIdentityStmt = this.#db.prepare("UPDATE auth_credentials SET identity_key = ? WHERE id = ?"); + try { + for (const row of rows) { + const identityKey = resolveRowCredentialIdentityKey(row.provider, row); + updateIdentityStmt.run(identityKey, row.id); + } + } finally { + updateIdentityStmt.finalize(); } } @@ -4927,9 +4960,13 @@ export class SqliteAuthCredentialStore implements AuthCredentialStore { updateAuthCredential(id: number, credential: AuthCredential): void { try { - const providerRow = this.#db.prepare("SELECT provider FROM auth_credentials WHERE id = ?").get(id) as - | { provider?: string } - | undefined; + const providerStmt = this.#db.prepare("SELECT provider FROM auth_credentials WHERE id = ?"); + let providerRow: { provider?: string } | undefined; + try { + providerRow = providerStmt.get(id) as { provider?: string } | undefined; + } finally { + providerStmt.finalize(); + } const provider = providerRow?.provider ?? ""; const serialized = serializeCredential(provider, credential); if (!serialized) return; @@ -5092,6 +5129,6 @@ export class SqliteAuthCredentialStore implements AuthCredentialStore { this.#upsertCacheStmt.finalize(); this.#deleteCachePrefixStmt.finalize(); this.#deleteExpiredCacheStmt.finalize(); - this.#db.close(); + this.#db.close(true); } } diff --git a/packages/ai/test/auth-storage-if-absent.test.ts b/packages/ai/test/auth-storage-if-absent.test.ts index 705bae9d21..2ccd2e6f69 100644 --- a/packages/ai/test/auth-storage-if-absent.test.ts +++ b/packages/ai/test/auth-storage-if-absent.test.ts @@ -136,6 +136,28 @@ describe("if-absent auth credential writes", () => { store.close(); } }); + test("SqliteAuthCredentialStore close releases WAL files for immediate directory removal", async () => { + const dbDir = path.join(tempDir, "close-releases-wal"); + await fs.mkdir(dbDir); + const store = await SqliteAuthCredentialStore.open(path.join(dbDir, "agent.db")); + + try { + const inserted = store.upsertAuthCredentialForProviderIfAbsent("anthropic", oauth("teardown")); + expect(inserted.inserted).toBe(true); + store.setCache("teardown-cache", "value", Math.floor(Date.now() / 1000) + 60); + expect(store.getCache("teardown-cache")).toBe("value"); + } finally { + store.close(); + } + + await fs.rm(dbDir, { recursive: true }); + expect( + await fs + .access(dbDir) + .then(() => true) + .catch(() => false), + ).toBe(false); + }); test("SqliteAuthCredentialStore returns skipped-invalid without inserting", async () => { const store = await SqliteAuthCredentialStore.open(path.join(tempDir, "invalid.db")); diff --git a/packages/coding-agent/CHANGELOG.md b/packages/coding-agent/CHANGELOG.md index 941674524f..8d54042a1e 100644 --- a/packages/coding-agent/CHANGELOG.md +++ b/packages/coding-agent/CHANGELOG.md @@ -24,6 +24,7 @@ - Slash commands now expand in non-interactive runs. `gjc -p "/init"` previously reached the model as the literal text `/init`, so no command body was injected, no file was written, and the model still answered as if the command had run. Print mode now loads the same bundled and file-based command list interactive mode uses before it prompts. - `gjc plugin install ` now names the marketplaces that offer `` when the npm resolution it falls back to fails, so a plugin name copied out of `gjc plugin discover` no longer dead-ends on a bare `install_failed`. - A remote multi-select ask now shows what is already selected. The ask tool re-issues one remote request per toggle, but the request carried no selection state, so Telegram kept posting an identical prompt with no sign that option 1 had been picked — the checkbox rendering existed only for durable workflow gates. `AskAnswerRequest` now carries `multi` and the selected option labels, the notification bus publishes them as `selectedOptionIndices` with the `(N selected)` question prefix while keeping the ask tool's own Next/Done control, and pre-numbered options (deep interview) are renumbered once instead of rendering as `1. ☑ 1. …`. +- Repaired Windows memory-release regressions in the v0.12.12 profiling and runtime paths: authenticated perf-corpus execution now recovers the real Windows command line, avoids reused child PIDs, revalidates sealed input bytes across Windows metadata APIs, and documents only the supported canonical runner; SQLite-backed settings and memory-guard claims finalize statements and close owned databases; checkpoint durability tolerates only Windows directory-`fsync` `EPERM`; recovery promotion is failure-atomic; and post-ACK team cutover failures consume the durable retry budget instead of looping ambiguously. ## [0.12.12] - 2026-08-05 diff --git a/packages/coding-agent/bench/perf-corpus-rlm-analysis.py b/packages/coding-agent/bench/perf-corpus-rlm-analysis.py index 7d425d49c7..ad3302e539 100644 --- a/packages/coding-agent/bench/perf-corpus-rlm-analysis.py +++ b/packages/coding-agent/bench/perf-corpus-rlm-analysis.py @@ -237,6 +237,11 @@ def _read_file_bytes(path: Path, maximum_bytes: int) -> tuple[bytes, os.stat_res return bytes(raw), before +def _rescan_file_identity(info: os.stat_result) -> tuple[int, ...]: + identity = (info.st_dev, info.st_ino, info.st_size, info.st_mtime_ns) + return identity if os.name == "nt" else (*identity, info.st_ctime_ns) + + def _canonical_digest(value: Any) -> str: return _sha256_bytes( json.dumps(value, ensure_ascii=False, allow_nan=False, sort_keys=True, separators=(",", ":")).encode("utf-8") @@ -2070,6 +2075,11 @@ def run_analysis( authenticated_file_stats, ) = _validate_sealed_inputs(input_real, prereg, expected_bindings) hashes.update(sealed_hashes) + authenticated_file_hashes = { + ATTEMPT_LEDGER_FILENAME: sealed_hashes["attemptLedgerSha256"], + RAW_MANIFEST_FILENAME: sealed_hashes["rawManifestSha256"], + **{filename: binding["sha256"] for filename, binding in raw_bindings.items()}, + } except (EvidenceError, FileNotFoundError) as error: finding = _finding( "SEALED_INPUT_INVALID", @@ -2113,7 +2123,7 @@ def run_analysis( entry_info: dict[str, os.stat_result] = {} for entry in sorted(scanned_entries, key=lambda item: item.name): try: - info = entry.stat(follow_symlinks=False) + info = (input_real / entry.name).lstat() scanned_total_size += info.st_size entry_info[entry.name] = info except OSError as error: @@ -2127,26 +2137,23 @@ def run_analysis( continue present_names.add(entry.name) authenticated_info = authenticated_file_stats.get(entry.name) - if authenticated_info is not None and ( - info.st_dev, - info.st_ino, - info.st_size, - info.st_mtime_ns, - info.st_ctime_ns, - ) != ( - authenticated_info.st_dev, - authenticated_info.st_ino, - authenticated_info.st_size, - authenticated_info.st_mtime_ns, - authenticated_info.st_ctime_ns, - ): - global_findings.append( - _finding( - "AUTHENTICATED_INPUT_METADATA_DRIFT", - "PROTOCOL", - f"authenticated input changed after byte capture: {entry.name}", + if authenticated_info is not None: + authenticated_input_changed = _rescan_file_identity(info) != _rescan_file_identity(authenticated_info) + try: + current_raw, _ = _read_file_bytes(input_real / entry.name, bounds["maximumBytesPerFile"]) + authenticated_input_changed = authenticated_input_changed or ( + _sha256_bytes(current_raw) != authenticated_file_hashes[entry.name] + ) + except (EvidenceError, OSError): + authenticated_input_changed = True + if authenticated_input_changed: + global_findings.append( + _finding( + "AUTHENTICATED_INPUT_METADATA_DRIFT", + "PROTOCOL", + f"authenticated input changed after byte capture: {entry.name}", + ) ) - ) if entry.name not in expected_names or not entry.name.endswith(".json"): global_findings.append( _finding( diff --git a/packages/coding-agent/bench/perf-corpus.bench.ts b/packages/coding-agent/bench/perf-corpus.bench.ts index 201cd1ee7c..cf06d7ed32 100644 --- a/packages/coding-agent/bench/perf-corpus.bench.ts +++ b/packages/coding-agent/bench/perf-corpus.bench.ts @@ -14,6 +14,7 @@ import * as childProcess from "node:child_process"; import * as fs from "node:fs"; import * as path from "node:path"; import * as url from "node:url"; +import { dlopen, FFIType, ptr, read, type Pointer } from "bun:ffi"; import { APPLIED_PERF_THRESHOLDS } from "./perf-threshold.ledger"; import { createMemoryBaselineWorkloads, type MemoryWorkload, workloadIterations } from "./memory-baseline-workloads"; import { @@ -71,6 +72,7 @@ const CANONICAL_RUNNER_EXEC_ARGV: readonly (readonly string[])[] = [ ["--expose-gc"], ["--smol", "--expose-gc"], ]; +const MAXIMUM_DISTINCT_CHILD_PID_ATTEMPTS = 3; function isCanonicalRunnerExecArgv(value: readonly string[]): boolean { return CANONICAL_RUNNER_EXEC_ARGV.some( @@ -78,7 +80,45 @@ function isCanonicalRunnerExecArgv(value: readonly string[]): boolean { ); } +function windowsWideString(value: Pointer): string { + const codeUnits: number[] = []; + for (let offset = 0; offset < 32_767 * 2; offset += 2) { + const codeUnit = read.u16(value, offset); + if (codeUnit === 0) return String.fromCharCode(...codeUnits); + codeUnits.push(codeUnit); + } + throw new Error("kernel process arguments unavailable"); +} + +function windowsProcessArguments(): string[] { + const kernel32 = dlopen("kernel32.dll", { + GetCommandLineW: { args: [], returns: FFIType.ptr }, + LocalFree: { args: [FFIType.ptr], returns: FFIType.ptr }, + }); + const shell32 = dlopen("shell32.dll", { + CommandLineToArgvW: { args: [FFIType.ptr, FFIType.ptr], returns: FFIType.ptr }, + }); + let argumentsPointer: Pointer | null = null; + try { + const argumentCount = new Uint32Array(1); + const commandLine = kernel32.symbols.GetCommandLineW(); + argumentsPointer = shell32.symbols.CommandLineToArgvW(commandLine, ptr(argumentCount)); + if (!argumentsPointer || argumentCount[0] === 0 || argumentCount[0] > 32_767) { + throw new Error("kernel process arguments unavailable"); + } + const parsedArgumentsPointer = argumentsPointer; + return Array.from({ length: argumentCount[0] }, (_, index) => + windowsWideString(read.ptr(parsedArgumentsPointer, index * 8) as Pointer), + ); + } finally { + if (argumentsPointer) kernel32.symbols.LocalFree(argumentsPointer); + shell32.close(); + kernel32.close(); + } +} + function kernelProcessArguments(): string[] { + if (process.platform === "win32") return windowsProcessArguments(); if (process.platform === "linux") { return fs .readFileSync(`/proc/${process.pid}/cmdline`, "utf8") @@ -559,28 +599,54 @@ function isolatedMemoryEntry(surface: MemorySurface): string { } return import.meta.path; } +export function spawnWithDistinctChildPid( + seenChildPids: ReadonlySet, + spawn: () => T, + maximumAttempts = MAXIMUM_DISTINCT_CHILD_PID_ATTEMPTS, +): T { + if (!Number.isSafeInteger(maximumAttempts) || maximumAttempts < 1) { + throw new Error("memory baseline child PID retry limit must be a positive integer"); + } + for (let attempt = 0; attempt < maximumAttempts; attempt++) { + const child = spawn(); + if (!Number.isSafeInteger(child.childPid) || child.childPid <= 0) { + throw new Error("memory baseline child returned an invalid PID"); + } + if (!seenChildPids.has(child.childPid)) return child; + } + throw new Error(`memory baseline child PID was reused after ${maximumAttempts} attempts`); +} + function buildIsolatedMemoryFixtures( profile: MemoryWorkloadProfile, targetDurationMs: number, memorySurfaceOrder: readonly MemorySurface[], ): PerfCorpusFixtureResult[] { + const seenChildPids = new Set(); return memorySurfaceOrder.map((surface, ordinal) => { - const result = Bun.spawnSync([process.execPath, "--smol", "--expose-gc", isolatedMemoryEntry(surface), MEMORY_CHILD_ARGUMENT], { - env: { - ...process.env, - GJC_MEMORY_CHILD_SURFACE: surface, - GJC_MEMORY_PROFILE: profile, - GJC_MEMORY_DURATION_MS: String(targetDurationMs), - GJC_MEMORY_SURFACE_ORDINAL: String(ordinal), - }, + const child = spawnWithDistinctChildPid(seenChildPids, () => { + const result = Bun.spawnSync([process.execPath, "--smol", "--expose-gc", isolatedMemoryEntry(surface), MEMORY_CHILD_ARGUMENT], { + env: { + ...process.env, + GJC_MEMORY_CHILD_SURFACE: surface, + GJC_MEMORY_PROFILE: profile, + GJC_MEMORY_DURATION_MS: String(targetDurationMs), + GJC_MEMORY_SURFACE_ORDINAL: String(ordinal), + }, + }); + if (result.exitCode !== 0) { + throw new Error( + `memory baseline child failed for ${surface}: ${new TextDecoder().decode(result.stderr).trim()}`, + ); + } + const fixture = JSON.parse(new TextDecoder().decode(result.stdout)) as PerfCorpusFixtureResult; + const childPid = fixture.memoryBaseline?.childPid; + if (childPid === undefined) throw new Error(`memory baseline child omitted PID for ${surface}`); + return { childPid, fixture }; }); - if (result.exitCode !== 0) { - throw new Error( - `memory baseline child failed for ${surface}: ${new TextDecoder().decode(result.stderr).trim()}`, - ); - } - return JSON.parse(new TextDecoder().decode(result.stdout)) as PerfCorpusFixtureResult; + seenChildPids.add(child.childPid); + return child.fixture; }); } @@ -724,7 +790,7 @@ function computePerfCorpusBenchmark( return report; } -export function runPerfCorpusBenchmark(options: { isolatedMemory?: boolean } = {}): PerfCorpusReport { +function runCanonicalPerfCorpusBenchmark(options: { isolatedMemory?: boolean } = {}): PerfCorpusReport { return computePerfCorpusBenchmark(authenticateCanonicalRunnerEntrypoint(), options); } @@ -737,7 +803,7 @@ if (CANONICAL_RUNNER_MODULE_MAIN) { if (!workload) throw new Error(`memory baseline workload unavailable for ${childSurface}`); process.stdout.write(`${JSON.stringify(buildMemoryFixture(workload, profile, durationTargetMs))}\n`); } else { - const report = runPerfCorpusBenchmark({ isolatedMemory: true }); + const report = runCanonicalPerfCorpusBenchmark({ isolatedMemory: true }); process.stdout.write(`${JSON.stringify(report, null, 2)}\n`); } } diff --git a/packages/coding-agent/src/gjc-runtime/memory-guard-owner-claims.ts b/packages/coding-agent/src/gjc-runtime/memory-guard-owner-claims.ts index ebee6c54e2..28c31fb212 100644 --- a/packages/coding-agent/src/gjc-runtime/memory-guard-owner-claims.ts +++ b/packages/coding-agent/src/gjc-runtime/memory-guard-owner-claims.ts @@ -1,4 +1,4 @@ -import { Database } from "bun:sqlite"; +import { Database, type Statement } from "bun:sqlite"; import * as fs from "node:fs/promises"; import { type LinuxProcPidProbeResult, probeLinuxProcPid } from "./linux-proc"; import { assertSafePathComponent } from "./session-layout"; @@ -133,13 +133,31 @@ async function openClaimsDatabase( const paths = memoryGuardClaimPaths(stateDir, sessionId); await prepareClaimDirectory(paths.root); const database = new Database(paths.databaseFile, { create: true }); - configureClaimsDatabase(database); - await enforceDatabaseModes(paths.databaseFile); - return { database, databaseFile: paths.databaseFile }; + try { + configureClaimsDatabase(database); + await enforceDatabaseModes(paths.databaseFile); + return { database, databaseFile: paths.databaseFile }; + } catch (error) { + database.close(true); + throw error; + } +} + +function withClaimStatement(database: Database, query: string, operation: (statement: Statement) => T): T { + const statement = database.prepare(query); + try { + return operation(statement); + } finally { + statement.finalize(); + } } function readEpoch(database: Database): number { - const row = database.prepare("SELECT value FROM meta WHERE key = 'epoch'").get() as { value: string } | null; + const row = withClaimStatement( + database, + "SELECT value FROM meta WHERE key = 'epoch'", + statement => statement.get() as { value: string } | null, + ); if (!row || !/^\d+$/.test(row.value)) throw new Error("memory_guard_claim_epoch_invalid"); const value = Number(row.value); if (!Number.isSafeInteger(value) || value < 0) throw new Error("memory_guard_claim_epoch_invalid"); @@ -149,16 +167,18 @@ function readEpoch(database: Database): number { function allocateEpoch(database: Database): number { const next = readEpoch(database) + 1; if (!Number.isSafeInteger(next) || next <= 0) throw new Error("memory_guard_claim_epoch_overflow"); - database.prepare("UPDATE meta SET value = ? WHERE key = 'epoch'").run(String(next)); + withClaimStatement(database, "UPDATE meta SET value = ? WHERE key = 'epoch'", statement => + statement.run(String(next)), + ); return next; } function readClaimRows(database: Database): PersistedMemoryGuardClaimRow[] { - const rows = database - .prepare( - "SELECT resource, epoch, session_id, generation, run_id, child_token, pid, process_start_time, tty_device, acquired_at FROM claims ORDER BY resource ASC", - ) - .all() as PersistedMemoryGuardClaimRow[]; + const rows = withClaimStatement( + database, + "SELECT resource, epoch, session_id, generation, run_id, child_token, pid, process_start_time, tty_device, acquired_at FROM claims ORDER BY resource ASC", + statement => statement.all() as PersistedMemoryGuardClaimRow[], + ); for (const row of rows) { if (!MEMORY_GUARD_CLAIM_RESOURCES.includes(row.resource)) throw new Error("memory_guard_claim_resource_invalid"); if (!Number.isSafeInteger(row.epoch) || row.epoch <= 0) throw new Error("memory_guard_claim_epoch_invalid"); @@ -208,22 +228,23 @@ function insertClaimRow( owner: MemoryGuardClaimOwner, acquiredAt: string, ): void { - database - .prepare( - "INSERT INTO claims(resource, epoch, session_id, generation, run_id, child_token, pid, process_start_time, tty_device, acquired_at) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?)", - ) - .run( - resource, - epoch, - owner.sessionId, - owner.generation, - owner.runId, - owner.childToken, - owner.pid, - owner.processStartTime, - owner.ttyDevice, - acquiredAt, - ); + withClaimStatement( + database, + "INSERT INTO claims(resource, epoch, session_id, generation, run_id, child_token, pid, process_start_time, tty_device, acquired_at) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?)", + statement => + statement.run( + resource, + epoch, + owner.sessionId, + owner.generation, + owner.runId, + owner.childToken, + owner.pid, + owner.processStartTime, + owner.ttyDevice, + acquiredAt, + ), + ); } function deleteExactClaimRow( @@ -232,21 +253,22 @@ function deleteExactClaimRow( epoch: number, owner: MemoryGuardClaimOwner, ): number { - return database - .prepare( - "DELETE FROM claims WHERE resource = ? AND epoch = ? AND session_id = ? AND generation = ? AND run_id = ? AND child_token = ? AND pid = ? AND process_start_time = ? AND tty_device = ?", - ) - .run( - resource, - epoch, - owner.sessionId, - owner.generation, - owner.runId, - owner.childToken, - owner.pid, - owner.processStartTime, - owner.ttyDevice, - ).changes; + return withClaimStatement( + database, + "DELETE FROM claims WHERE resource = ? AND epoch = ? AND session_id = ? AND generation = ? AND run_id = ? AND child_token = ? AND pid = ? AND process_start_time = ? AND tty_device = ?", + statement => + statement.run( + resource, + epoch, + owner.sessionId, + owner.generation, + owner.runId, + owner.childToken, + owner.pid, + owner.processStartTime, + owner.ttyDevice, + ).changes, + ); } function rollbackQuietly(database: Database): void { @@ -299,7 +321,7 @@ export async function acquireMemoryGuardClaims( } throw new Error("memory_guard_claim_rows_changed"); } finally { - database.close(); + database.close(true); } } @@ -320,7 +342,7 @@ export async function releaseMemoryGuardClaims(stateDir: string, claim: MemoryGu rollbackQuietly(database); throw error; } finally { - database.close(); + database.close(true); } } @@ -355,6 +377,7 @@ export async function probeMemoryGuardClaimsReleased( database.exec("COMMIT"); await enforceDatabaseModes(databaseFile); const proof = issueMemoryGuardClaimsLease({ writerEpoch, ttyEpoch, owner, claimStorePath: databaseFile }); + database.close(true); await releaseMemoryGuardClaims(stateDir, proof); return proof; } catch (error) { @@ -364,7 +387,7 @@ export async function probeMemoryGuardClaimsReleased( } throw new Error("memory_guard_claim_rows_changed"); } finally { - database.close(); + database.close(true); } } @@ -384,6 +407,6 @@ export async function readMemoryGuardClaimsForTest( claims: readClaimRows(database), }; } finally { - database.close(); + database.close(true); } } diff --git a/packages/coding-agent/src/gjc-runtime/team-runtime.ts b/packages/coding-agent/src/gjc-runtime/team-runtime.ts index 246b142cc3..397e50a9a6 100644 --- a/packages/coding-agent/src/gjc-runtime/team-runtime.ts +++ b/packages/coding-agent/src/gjc-runtime/team-runtime.ts @@ -629,6 +629,18 @@ export interface GjcTeamRuntimeTestSeams { ) => { exitCode?: number } | Promise<{ exitCode?: number }>; continuationBeforeDispatch?: () => Promise; continuationAckPoll?: () => Promise; + memoryGuardReplacementPhase?: ( + phase: + | "post_ack_authority" + | "successor_pane_recheck" + | "heartbeat_publication" + | "config_publication" + | "lifecycle_publication" + | "predecessor_termination" + | "rollback_successor_termination" + | "rollback_predecessor_restoration", + ) => void | Promise; + memoryGuardHostPlatform?: NodeJS.Platform; } let gjcTeamRuntimeTestSeams: GjcTeamRuntimeTestSeams | undefined; @@ -1697,12 +1709,16 @@ async function applyWorkerMemoryGuardUnlocked(input: { let ledger = ledgers.get(worker.id) ?? (await readWorkerMemoryGuardLedger(dir, worker.id, input.platform)); const nowIso = now(); const currentTaskId = task?.id; - const incidentId = input.incidentId ?? stableHash(`${worker.id}:${currentTaskId ?? "none"}:${nowIso}`).slice(0, 16); + const memoryGuardHostPlatform = gjcTeamRuntimeTestSeams?.memoryGuardHostPlatform ?? process.platform; + const incidentId = + input.incidentId ?? + (ledger.retry_count > 0 ? ledger.last_incident_id : undefined) ?? + stableHash(`${worker.id}:${currentTaskId ?? "none"}:${nowIso}`).slice(0, 16); const baseReason = input.reason?.trim() || "memory_guard_requested"; if (!authority.valid || !Number.isFinite(leaseExpiresAt) || leaseExpiresAt <= currentTimeMs()) { const noClaimReason = - process.platform !== "linux" || input.platform !== "linux" - ? `unsupported_platform:${input.platform}:host:${process.platform}:${baseReason}` + memoryGuardHostPlatform !== "linux" || input.platform !== "linux" + ? `unsupported_platform:${input.platform}:host:${memoryGuardHostPlatform}:${baseReason}` : "worker_has_no_exact_active_claim"; ledger = { ...ledger, @@ -1754,14 +1770,14 @@ async function applyWorkerMemoryGuardUnlocked(input: { automatic_action_allowed: input.allowAutomaticAction, updated_at: nowIso, }; - if (process.platform !== "linux" || input.platform !== "linux") { + if (memoryGuardHostPlatform !== "linux" || input.platform !== "linux") { ledger = { ...ledger, platform: input.platform, state: "advisory", current_task_id: currentTaskId, last_incident_id: incidentId, - last_reason: `unsupported_platform:${input.platform}:host:${process.platform}:${baseReason}`, + last_reason: `unsupported_platform:${input.platform}:host:${memoryGuardHostPlatform}:${baseReason}`, last_pid_probe: input.pidProbe, updated_at: nowIso, }; @@ -1916,7 +1932,7 @@ async function applyWorkerMemoryGuardUnlocked(input: { newPaneId = await relaunchWorkerPaneForMemoryGuard({ config, worker, - platform: process.platform, + platform: memoryGuardHostPlatform, startupAckPath, replacementToken, startupAckTimeoutMs, @@ -1966,48 +1982,10 @@ async function applyWorkerMemoryGuardUnlocked(input: { ledger, }; } - const postAckInventory = await readGjcContinuationAuthorityInventory(dir); - const postAckAuthority = postAckInventory.valid - ? selectGjcContinuationWorkerAuthority(postAckInventory, worker.id) - : { valid: false as const, taskCount: 0, claimCount: 0 }; - if ( - !postAckAuthority.valid || - postAckAuthority.task.id !== task?.id || - postAckAuthority.claim.token !== task?.claim?.token || - postAckAuthority.claim.leased_until !== task?.claim?.leased_until || - Date.parse(postAckAuthority.claim.leased_until) <= currentTimeMs() - ) { - executeTeamTmuxMutation(config, { type: "kill-pane", paneId: newPaneId }); - await restorePredecessorStartupState(); - return { - ok: true, - result: "advisory", - lifecycle_mutated: false, - reason: "worker_claim_authority_changed_after_startup", - }; - } - const successorPane = probePaneTeamTarget(config, newPaneId); - if (!successorPane.exists || !successorPane.belongsToTeamTarget || !successorPane.pid) { - executeTeamTmuxMutation(config, { type: "kill-pane", paneId: newPaneId }); - await restorePredecessorStartupState(); - return { - ok: true, - result: "advisory", - lifecycle_mutated: false, - reason: "successor_pane_unavailable_after_startup", - }; - } const heartbeatPath = path.join(dir, "workers", safePathSegment("worker_id", worker.id), "heartbeat.json"); const previousHeartbeat = (await Bun.file(heartbeatPath).exists()) ? await Bun.file(heartbeatPath).text() : undefined; - const successorHeartbeat: WorkerHeartbeatFile = { - pid: successorPane.pid, - last_turn_at: now(), - turn_count: 0, - alive: true, - process_start_time: await readLinuxProcessStartTime(successorPane.pid), - }; const nextConfig: GjcTeamConfig = { ...config, workers: config.workers.map(candidate => @@ -2017,35 +1995,144 @@ async function applyWorkerMemoryGuardUnlocked(input: { ), updated_at: nowIso, }; - const rollbackReplacement = async (): Promise => { - executeTeamTmuxMutation(config, { type: "kill-pane", paneId: newPaneId }); - if (previousHeartbeat === undefined) await fs.rm(heartbeatPath, { force: true }); - else await Bun.write(heartbeatPath, previousHeartbeat); - await syncTeamConfigAndManifest(dir, config); - await restorePredecessorStartupState(); + let predecessorTerminationStarted = false; + const rollbackReplacement = async () => { + const failures: string[] = []; + let successorTerminated = false; + let predecessorStateRestored = false; + try { + await gjcTeamRuntimeTestSeams?.memoryGuardReplacementPhase?.("rollback_successor_termination"); + executeTeamTmuxMutation(config, { type: "kill-pane", paneId: newPaneId }); + successorTerminated = true; + } catch (error) { + failures.push(error instanceof Error && error.message ? error.message : "successor_termination_failed"); + } + if (predecessorTerminationStarted) { + failures.push("predecessor_termination_ambiguous"); + } else { + try { + await gjcTeamRuntimeTestSeams?.memoryGuardReplacementPhase?.("rollback_predecessor_restoration"); + if (previousHeartbeat === undefined) await fs.rm(heartbeatPath, { force: true }); + else await Bun.write(heartbeatPath, previousHeartbeat); + await syncTeamConfigAndManifest(dir, config); + await restorePredecessorStartupState(); + predecessorStateRestored = true; + } catch (error) { + failures.push(error instanceof Error && error.message ? error.message : "predecessor_restoration_failed"); + } + } + return { + successor_terminated: successorTerminated, + predecessor_state_restored: predecessorStateRestored, + ...(failures.length ? { reason: failures.join(";") } : {}), + }; + }; + const finalizePostAckFailure = async (error: unknown): Promise> => { + const primaryReason = + error instanceof Error && error.message + ? `memory_guard_replacement_commit_failed:${error.message}` + : "memory_guard_replacement_commit_failed"; + const rollback = await rollbackReplacement(); + const ambiguousRollback = !rollback.successor_terminated || !rollback.predecessor_state_restored; + const reason = rollback.reason ? `${primaryReason};rollback_failed:${rollback.reason}` : primaryReason; + const retried = withMemoryGuardRetry(ledger, { + platform: input.platform, + reason, + incidentId, + currentTaskId, + pidProbe: input.pidProbe, + nowIso, + }); + ledger = { + ...retried.ledger, + ...(ambiguousRollback ? { state: "blocked" as const, retry_count: ledger.retry_limit } : {}), + last_replacement: { + old_pane_id: worker.pane_id, + new_pane_id: newPaneId, + recorded_at: nowIso, + rollback, + }, + }; + await writeWorkerMemoryGuardLedger(dir, ledger); + const finalBlocked = retried.finalBlocked || ambiguousRollback; + if (finalBlocked) + await input.withTaskMutation(taskMutation => + finalizeWorkerMemoryGuardBlockedState({ + teamName: input.teamName, + dir, + worker, + task, + taskMutation, + reason, + cwd: input.cwd, + env: input.env, + }), + ); + await appendWorkerMemoryGuardAction({ + dir, + teamName: input.teamName, + cwd: input.cwd, + workerId: worker.id, + task, + incidentId, + action: finalBlocked ? "blocked" : "replace", + result: finalBlocked ? "blocked" : "failed", + reason, + }); + return { + ok: true, + result: finalBlocked ? "blocked" : "retrying", + lifecycle_mutated: finalBlocked, + ledger, + }; }; try { + await gjcTeamRuntimeTestSeams?.memoryGuardReplacementPhase?.("post_ack_authority"); + const postAckInventory = await readGjcContinuationAuthorityInventory(dir); + const postAckAuthority = postAckInventory.valid + ? selectGjcContinuationWorkerAuthority(postAckInventory, worker.id) + : { valid: false as const, taskCount: 0, claimCount: 0 }; + if ( + !postAckAuthority.valid || + postAckAuthority.task.id !== task?.id || + postAckAuthority.claim.token !== task?.claim?.token || + postAckAuthority.claim.leased_until !== task?.claim?.leased_until || + Date.parse(postAckAuthority.claim.leased_until) <= currentTimeMs() + ) + throw new Error("worker_claim_authority_changed_after_startup"); + await gjcTeamRuntimeTestSeams?.memoryGuardReplacementPhase?.("successor_pane_recheck"); + const successorPane = probePaneTeamTarget(config, newPaneId); + if (!successorPane.exists || !successorPane.belongsToTeamTarget || !successorPane.pid) + throw new Error("successor_pane_unavailable_after_startup"); + const successorHeartbeat: WorkerHeartbeatFile = { + pid: successorPane.pid, + last_turn_at: now(), + turn_count: 0, + alive: true, + process_start_time: await readLinuxProcessStartTime(successorPane.pid), + }; + await gjcTeamRuntimeTestSeams?.memoryGuardReplacementPhase?.("heartbeat_publication"); await writeJsonFile(heartbeatPath, successorHeartbeat); + await gjcTeamRuntimeTestSeams?.memoryGuardReplacementPhase?.("config_publication"); await syncTeamConfigAndManifest(dir, nextConfig); + await gjcTeamRuntimeTestSeams?.memoryGuardReplacementPhase?.("lifecycle_publication"); await writeLifecycleRecord(workerRuntime, dir, { ...worker, pane_id: newPaneId }, "ready", { pane_id: newPaneId, started_at: nowIso, stop_reason: undefined, stopped_at: undefined, }); + await gjcTeamRuntimeTestSeams?.memoryGuardReplacementPhase?.("successor_pane_recheck"); const cutoverPane = probePaneTeamTarget(config, newPaneId); if (!cutoverPane.exists || !cutoverPane.belongsToTeamTarget || cutoverPane.pid !== successorHeartbeat.pid) throw new Error(`memory_guard_successor_pane_changed:${worker.id}`); if (worker.pane_id && !config.dry_run) { + predecessorTerminationStarted = true; + await gjcTeamRuntimeTestSeams?.memoryGuardReplacementPhase?.("predecessor_termination"); executeTeamTmuxMutation(config, { type: "kill-pane", paneId: worker.pane_id }); } } catch (error) { - await rollbackReplacement(); - throw new Error( - error instanceof Error && error.message - ? `memory_guard_replacement_commit_failed:${error.message}` - : "memory_guard_replacement_commit_failed", - ); + return finalizePostAckFailure(error); } ledger = { ...ledger, diff --git a/packages/coding-agent/src/gjc-runtime/team-worker-memory-guard.ts b/packages/coding-agent/src/gjc-runtime/team-worker-memory-guard.ts index 01a360e1e0..c513349696 100644 --- a/packages/coding-agent/src/gjc-runtime/team-worker-memory-guard.ts +++ b/packages/coding-agent/src/gjc-runtime/team-worker-memory-guard.ts @@ -28,6 +28,11 @@ export interface GjcTeamWorkerMemoryGuardReplacement { old_pane_id?: string; new_pane_id?: string; recorded_at: string; + rollback?: { + successor_terminated: boolean; + predecessor_state_restored: boolean; + reason?: string; + }; } export interface GjcTeamWorkerMemoryGuardLedger { @@ -81,7 +86,7 @@ const ledgerKeys = new Set([ "updated_at", ]); const checkpointKeys = new Set(["kind", "files", "head", "commit", "recorded_at"]); -const replacementKeys = new Set(["old_pane_id", "new_pane_id", "recorded_at"]); +const replacementKeys = new Set(["old_pane_id", "new_pane_id", "recorded_at", "rollback"]); const absentPidProbeKeys = new Set(["kind"]); const livePidProbeKeys = new Set(["kind", "start_time"]); const unverifiablePidProbeKeys = new Set(["kind", "reason"]); @@ -156,6 +161,16 @@ export function isCanonicalGjcTeamWorkerMemoryGuardCheckpoint( ); } +function isCanonicalGjcTeamWorkerMemoryGuardReplacementRollback(value: unknown): boolean { + return ( + isRecord(value) && + hasExactKeys(value, new Set(["successor_terminated", "predecessor_state_restored", "reason"])) && + typeof value.successor_terminated === "boolean" && + typeof value.predecessor_state_restored === "boolean" && + (value.reason === undefined || isNonEmptyString(value.reason)) + ); +} + export function isCanonicalGjcTeamWorkerMemoryGuardReplacement( value: unknown, ): value is GjcTeamWorkerMemoryGuardReplacement { @@ -164,7 +179,8 @@ export function isCanonicalGjcTeamWorkerMemoryGuardReplacement( hasExactKeys(value, replacementKeys) && (value.old_pane_id === undefined || isNonEmptyString(value.old_pane_id)) && (value.new_pane_id === undefined || isNonEmptyString(value.new_pane_id)) && - isTimestamp(value.recorded_at) + isTimestamp(value.recorded_at) && + (value.rollback === undefined || isCanonicalGjcTeamWorkerMemoryGuardReplacementRollback(value.rollback)) ); } diff --git a/packages/coding-agent/src/session/agent-storage.ts b/packages/coding-agent/src/session/agent-storage.ts index bc80f63f66..23eae41bee 100644 --- a/packages/coding-agent/src/session/agent-storage.ts +++ b/packages/coding-agent/src/session/agent-storage.ts @@ -94,7 +94,13 @@ CREATE TABLE IF NOT EXISTS model_usage ( CREATE TABLE IF NOT EXISTS schema_version (version INTEGER PRIMARY KEY); `); - const settingsInfo = this.#db.prepare("PRAGMA table_info(settings)").all() as Array<{ name?: string }>; + const settingsInfoStmt = this.#db.prepare("PRAGMA table_info(settings)"); + let settingsInfo: Array<{ name?: string }>; + try { + settingsInfo = settingsInfoStmt.all() as Array<{ name?: string }>; + } finally { + settingsInfoStmt.finalize(); + } const hasSettingsTable = settingsInfo.length > 0; const hasKey = settingsInfo.some(column => column.name === "key"); const hasValue = settingsInfo.some(column => column.name === "value"); @@ -110,7 +116,13 @@ CREATE TABLE settings ( } else if (!hasKey || !hasValue) { // Migrate v1 schema: single JSON blob in `data` column → per-key rows let legacySettings: Record | null = null; - const row = this.#db.prepare("SELECT data FROM settings WHERE id = 1").get() as { data?: string } | undefined; + const legacySettingsStmt = this.#db.prepare("SELECT data FROM settings WHERE id = 1"); + let row: { data?: string } | undefined; + try { + row = legacySettingsStmt.get() as { data?: string } | undefined; + } finally { + legacySettingsStmt.finalize(); + } if (row?.data) { try { const parsed = JSON.parse(row.data); @@ -137,11 +149,15 @@ CREATE TABLE settings ( const insert = this.#db.prepare( `INSERT INTO settings (key, value, updated_at) VALUES (?, ?, ${SQLITE_NOW_EPOCH})`, ); - for (const [key, value] of Object.entries(settings)) { - if (value === undefined) continue; - const serialized = JSON.stringify(value); - if (serialized === undefined) continue; - insert.run(key, serialized); + try { + for (const [key, value] of Object.entries(settings)) { + if (value === undefined) continue; + const serialized = JSON.stringify(value); + if (serialized === undefined) continue; + insert.run(key, serialized); + } + } finally { + insert.finalize(); } } }); @@ -149,9 +165,13 @@ CREATE TABLE settings ( migrate(legacySettings); } - const versionRow = this.#db.prepare("SELECT version FROM schema_version ORDER BY version DESC LIMIT 1").get() as - | { version?: number } - | undefined; + const versionStmt = this.#db.prepare("SELECT version FROM schema_version ORDER BY version DESC LIMIT 1"); + let versionRow: { version?: number } | undefined; + try { + versionRow = versionStmt.get() as { version?: number } | undefined; + } finally { + versionStmt.finalize(); + } const schemaVersion = typeof versionRow?.version === "number" ? versionRow.version : 0; if (versionRow?.version !== undefined && versionRow.version !== SCHEMA_VERSION) { logger.warn("AgentStorage schema version mismatch", { @@ -162,7 +182,12 @@ CREATE TABLE settings ( if (schemaVersion < SCHEMA_VERSION) { this.#migrateSchema(schemaVersion); } - this.#db.prepare("INSERT OR REPLACE INTO schema_version(version) VALUES (?)").run(SCHEMA_VERSION); + const versionInsertStmt = this.#db.prepare("INSERT OR REPLACE INTO schema_version(version) VALUES (?)"); + try { + versionInsertStmt.run(SCHEMA_VERSION); + } finally { + versionInsertStmt.finalize(); + } } #migrateSchema(fromVersion: number): void { @@ -342,13 +367,18 @@ FROM model_usage_legacy ? "SELECT id, provider, credential_type, data, disabled_cause FROM auth_credentials WHERE provider = ? ORDER BY id ASC" : "SELECT id, provider, credential_type, data, disabled_cause FROM auth_credentials ORDER BY id ASC", ); - const rows = (provider ? stmt.all(provider) : stmt.all()) as Array<{ + let rows: Array<{ id: number; provider: string; credential_type: string; data: string; disabled_cause: string | null; }>; + try { + rows = (provider ? stmt.all(provider) : stmt.all()) as typeof rows; + } finally { + stmt.finalize(); + } const results: StoredAuthCredential[] = []; for (const row of rows) { diff --git a/packages/coding-agent/src/session/internal/managed-session-storage.ts b/packages/coding-agent/src/session/internal/managed-session-storage.ts index 7ef683dfb7..5338a8e514 100644 --- a/packages/coding-agent/src/session/internal/managed-session-storage.ts +++ b/packages/coding-agent/src/session/internal/managed-session-storage.ts @@ -7,6 +7,7 @@ import { applyOwnerOnlyPathSecurity, exactRemoveDirectoryTree, exactReplacePath, + exactRestore, exactUnlink, linkNoReplacePath, type NativeDirectoryTreeSnapshot, @@ -96,7 +97,7 @@ function publishFailure(outcome: NativePublishOutcome): ManagedPublishError { return new ManagedPublishError(classification, outcome); } -function exactUnlinkCompleted(result: NativeExactUnlinkResult): boolean { +export function exactUnlinkCompleted(result: NativeExactUnlinkResult): boolean { return ( result.ok || (result.code === "cleanup_pending" && @@ -1087,6 +1088,63 @@ export class ManagedSessionDescendantStore { } await publishManagedFileNoReplace(resolved, bytes, undefined, this.#root, this.#policy); } + /** + * Move an exact captured file to an absent name without reopening the source + * pathname as authority. The rename changes ctime, so its stable identity is + * rechecked after publication. + */ + moveExpectedNoReplace( + sourceRelativePath: string, + destinationRelativePath: string, + expected: ManagedFileSnapshot, + ): void { + this.#beforeMutation(); + this.#assertBound(); + const source = this.#resolve(sourceRelativePath); + const destination = this.#resolve(destinationRelativePath); + if (this.#authority) { + const moved = this.#authority.renameManagedFileNoReplace( + this.#relative(source), + this.#relative(destination), + expected.identity.dev.toString(), + expected.identity.ino.toString(), + expected.identity.size.toString(), + expected.identity.mtimeNs.toString(), + expected.identity.ctimeNs.toString(), + expected.identity.sha256, + ); + const outcome = classifyNativePublishOutcome(moved, "retained_file"); + if (!outcome.ok) throw publishFailure(outcome); + } else { + const sourceParent = path.dirname(source); + if (sourceParent !== path.dirname(destination)) throw new Error("managed_move_parent_mismatch"); + const parent = fs.lstatSync(sourceParent, { bigint: true }); + const moved = exactRestore(source, destination, { + dev: expected.identity.dev, + ino: expected.identity.ino, + nlink: expected.identity.nlink, + parentDev: parent.dev, + parentIno: parent.ino, + size: BigInt(expected.identity.size), + mtimeNs: expected.identity.mtimeNs, + sha256: expected.identity.sha256, + }); + if (!moved.ok) + throw new Error( + moved.code === "collision" ? "destination_conflict" : (moved.code ?? "managed_move_failed"), + ); + } + const published = this.readExpected(destinationRelativePath); + if ( + !published || + !sameReplacementIdentity(published.identity, expected.identity) || + published.identity.sha256 !== expected.identity.sha256 + ) { + throw new Error("managed_move_identity_mismatch"); + } + if (!this.#authority) fsyncDirectory(this.#baseDir); + this.#assertBound(); + } publishNoReplaceSync(relativePath: string, bytes: Uint8Array): void { this.#beforeMutation(); diff --git a/packages/coding-agent/src/session/session-manager.ts b/packages/coding-agent/src/session/session-manager.ts index 7967db927f..47c8eadfce 100644 --- a/packages/coding-agent/src/session/session-manager.ts +++ b/packages/coding-agent/src/session/session-manager.ts @@ -75,6 +75,7 @@ import { import { assertManagedDirectoryRoot, captureManagedFileNoFollow, + exactUnlinkCompleted, fsyncManagedArtifactTree, MANAGED_ARTIFACT_MAX_FILE_BYTES, type ManagedDirectoryRoot, @@ -644,6 +645,8 @@ async function fsyncDirectoryPath(directoryPath: string): Promise { const directory = await fs.promises.open(directoryPath, "r"); try { await directory.sync(); + } catch (error) { + if (process.platform !== "win32" || (error as NodeJS.ErrnoException | undefined)?.code !== "EPERM") throw error; } finally { await directory.close(); } @@ -4850,6 +4853,10 @@ export class SessionManager { #ensuredOnDisk: boolean = false; #recoveryHydrationContext: RecoveryHydrationContext | undefined; #recoveryPromotionTranscriptPath: string | undefined; + #recoverySourceRetirementCleanup: + | { readonly kind: "managed"; readonly relativePath: string; readonly snapshot: ManagedFileSnapshot } + | { readonly kind: "explicit"; readonly identity: ResumeSessionIdentity } + | undefined; #memoryGuardParticipantIngressToken: symbol | undefined; #fileEntries: FileEntry[] = []; #pendingStrictAdoption: @@ -7705,6 +7712,31 @@ export class SessionManager { } /** Close the persistent writer after flushing all pending data. */ + async #retryRecoverySourceRetirementCleanup(): Promise { + const cleanup = this.#recoverySourceRetirementCleanup; + if (!cleanup) return; + try { + if (cleanup.kind === "managed") + this.#managedTranscriptStore().removeExpected(cleanup.relativePath, cleanup.snapshot); + else { + const removed = native.exactUnlink(cleanup.identity.canonicalPath, { + dev: cleanup.identity.dev, + ino: cleanup.identity.ino, + size: BigInt(cleanup.identity.size), + mtimeNs: cleanup.identity.mtimeNs, + sha256: cleanup.identity.sha256, + quarantineName: `.gjc-recovery-retire-${process.pid}-${crypto.randomUUID()}`, + }); + if (!exactUnlinkCompleted(removed)) throw new Error(removed.code ?? "recovery_source_retire_failed"); + await fsyncDirectoryPath(path.dirname(cleanup.identity.canonicalPath)); + } + this.#recoverySourceRetirementCleanup = undefined; + } catch (error) { + logger.warn("Recovery source retirement retry failed; retaining exact cleanup authority", { + error: toError(error).message, + }); + } + } async close(): Promise { // Drain any uncommitted prepared successors before releasing resources so // dispose/shutdown retains exact cleanup authority (#3138). @@ -7715,6 +7747,7 @@ export class SessionManager { error: toError(error).message, }); } + await this.#retryRecoverySourceRetirementCleanup(); await this.#queuePersistTask(async () => { if (this.#persistWriter) { await this.#closePersistWriterInternal(); @@ -7750,6 +7783,7 @@ export class SessionManager { error: toError(error).message, }); } + await this.#retryRecoverySourceRetirementCleanup(); let outcome: SessionManagerCloseOutcome = { kind: "closed" }; await this.#queuePersistTask(async () => { const writer = this.#persistWriter; @@ -10654,17 +10688,174 @@ export class SessionManager { "retain-and-throw", ); try { + const promotionName = path.basename(promotionPath); + const stagingName = `.${promotionName}.${crypto.randomUUID()}.recovery-staging`; if (this.destination.kind === "managed") { const store = this.#managedTranscriptStore(); const sourceName = path.basename(context.identity.canonicalPath); const sourceSnapshot = store.readExpected(sourceName); if (!sourceSnapshot) throw new Error("Recovery transcript authority changed before publication."); - await store.publishNoReplace(path.basename(promotionPath), promotedSource.content); - store.removeExpected(sourceName, sourceSnapshot); + const intendedDigest = crypto.createHash("sha256").update(promotedSource.content).digest("hex"); + const existing = store.readExpected(promotionName); + if (existing) { + if ( + !existing.bytes.equals(Buffer.from(promotedSource.content)) || + existing.identity.sha256 !== intendedDigest + ) + throw new Error("Recovery successor authority conflicts with promotion."); + } else { + await store.publishNoReplace(stagingName, promotedSource.content); + const stagedSnapshot = store.readExpected(stagingName); + if (!stagedSnapshot) throw new Error("Recovery successor staging identity is unavailable."); + try { + store.moveExpectedNoReplace(stagingName, promotionName, stagedSnapshot); + } catch (error) { + const rollbackPublishedSuccessor = (relativePath: string, snapshot: ManagedFileSnapshot): void => { + try { + store.removeExpected(relativePath, snapshot); + } catch (cleanupError) { + logger.warn("Recovery successor rollback cleanup failed; original promotion error preserved", { + cleanupError: toError(cleanupError).message, + promotionError: toError(error).message, + }); + } + }; + const final = store.readExpected(promotionName); + if ( + final && + final.identity.dev === stagedSnapshot.identity.dev && + final.identity.ino === stagedSnapshot.identity.ino && + final.identity.size === stagedSnapshot.identity.size && + final.identity.mtimeNs === stagedSnapshot.identity.mtimeNs && + final.identity.sha256 === stagedSnapshot.identity.sha256 + ) + rollbackPublishedSuccessor(promotionName, final); + else { + const remaining = store.readExpected(stagingName); + if ( + remaining && + remaining.identity.dev === stagedSnapshot.identity.dev && + remaining.identity.ino === stagedSnapshot.identity.ino && + remaining.identity.size === stagedSnapshot.identity.size && + remaining.identity.mtimeNs === stagedSnapshot.identity.mtimeNs && + remaining.identity.sha256 === stagedSnapshot.identity.sha256 + ) + rollbackPublishedSuccessor(stagingName, stagedSnapshot); + } + throw error; + } + } + // A verified successor is retry-safe. Source-retirement failure retains a + // recoverable duplicate and does not wedge promotion. + try { + store.removeExpected(sourceName, sourceSnapshot); + } catch (error) { + this.#recoverySourceRetirementCleanup = { + kind: "managed", + relativePath: sourceName, + snapshot: sourceSnapshot, + }; + logger.warn("Recovery source retirement failed after successor commit; exact retry scheduled", { + error: toError(error).message, + }); + } } else { - await writeOwnerOnlyFileNoReplace(promotionPath, promotedSource.content); - await fs.promises.rm(context.identity.canonicalPath, { force: true }); - await fsyncDirectoryPath(path.dirname(context.identity.canonicalPath)); + const directory = path.dirname(promotionPath); + const intendedDigest = crypto.createHash("sha256").update(promotedSource.content).digest("hex"); + const retireSource = () => + native.exactUnlink(context.identity.canonicalPath, { + dev: context.identity.dev, + ino: context.identity.ino, + size: BigInt(context.identity.size), + mtimeNs: context.identity.mtimeNs, + sha256: context.identity.sha256, + quarantineName: `.gjc-recovery-retire-${process.pid}-${crypto.randomUUID()}`, + }); + const existing = inspectResumeSessionFile(promotionPath, this.storage); + if (!("kind" in existing)) { + if ( + !Buffer.from(existing.content).equals(Buffer.from(promotedSource.content)) || + existing.identity.sha256 !== intendedDigest + ) + throw new Error("Recovery successor authority conflicts with promotion."); + fsyncResumeSessionIdentity(existing.identity); + const retired = retireSource(); + if (exactUnlinkCompleted(retired)) + await fsyncDirectoryPath(path.dirname(context.identity.canonicalPath)); + else { + this.#recoverySourceRetirementCleanup = { kind: "explicit", identity: context.identity }; + logger.warn("Recovery source retirement failed after successor commit; exact retry scheduled", { + code: retired.code, + }); + } + } else { + const stagingPath = path.join(directory, stagingName); + await writeOwnerOnlyFileNoReplace(stagingPath, promotedSource.content); + const staged = await fs.promises.lstat(stagingPath, { bigint: true }); + const stagedIdentity = { + dev: staged.dev, + ino: staged.ino, + size: staged.size, + mtimeNs: staged.mtimeNs, + sha256: intendedDigest, + }; + let published = false; + try { + const outcome = classifyNativePublishOutcome(native.renameNoReplacePath(stagingPath, promotionPath)); + if (!outcome.ok) throw new Error(formatNativePublishDiagnostic(outcome)); + published = true; + const final = await fs.promises.lstat(promotionPath, { bigint: true }); + const finalDigest = crypto + .createHash("sha256") + .update(await fs.promises.readFile(promotionPath)) + .digest("hex"); + if ( + final.dev !== stagedIdentity.dev || + final.ino !== stagedIdentity.ino || + final.size !== stagedIdentity.size || + final.mtimeNs !== stagedIdentity.mtimeNs || + finalDigest !== stagedIdentity.sha256 + ) + throw new Error("Recovery successor final identity changed during promotion."); + await fsyncDirectoryPath(directory); + } catch (error) { + const rollbackPath = published ? promotionPath : stagingPath; + const candidate = await fs.promises.lstat(rollbackPath, { bigint: true }).catch(() => undefined); + if ( + candidate && + candidate.dev === stagedIdentity.dev && + candidate.ino === stagedIdentity.ino && + candidate.size === stagedIdentity.size && + candidate.mtimeNs === stagedIdentity.mtimeNs + ) { + const digest = crypto + .createHash("sha256") + .update(await fs.promises.readFile(rollbackPath)) + .digest("hex"); + if (digest === stagedIdentity.sha256) { + const removed = native.exactUnlink(rollbackPath, { + ...stagedIdentity, + quarantineName: `.gjc-recovery-rollback-${process.pid}-${crypto.randomUUID()}`, + }); + if (!exactUnlinkCompleted(removed)) + throw new Error(removed.code ?? "recovery_successor_rollback_failed", { cause: error }); + await fsyncDirectoryPath(directory); + } + } + throw error; + } + // A failed retirement retains a durable verified duplicate and is reconciled + // as success, leaving later exact cleanup to recovery. + const retired = retireSource(); + if (exactUnlinkCompleted(retired)) + await fsyncDirectoryPath(path.dirname(context.identity.canonicalPath)); + else { + this.#recoverySourceRetirementCleanup = { kind: "explicit", identity: context.identity }; + logger.warn("Recovery source retirement failed after successor commit; exact retry scheduled", { + code: retired.code, + }); + } + } } } catch (error) { transition.dispose(); diff --git a/packages/coding-agent/test/gjc-runtime/memory-guard-owner-claims.test.ts b/packages/coding-agent/test/gjc-runtime/memory-guard-owner-claims.test.ts index 899f608c28..8c741e000b 100644 --- a/packages/coding-agent/test/gjc-runtime/memory-guard-owner-claims.test.ts +++ b/packages/coding-agent/test/gjc-runtime/memory-guard-owner-claims.test.ts @@ -211,10 +211,11 @@ describe("memory guard owner claims", () => { } }); - it("proves claims released by fencing epochs and exact-releasing them", async () => { + it("proves claims released by fencing epochs, exact-releasing them, and closing database handles", async () => { const stateDir = await tempStateDir(); const probeOwner = owner({ childToken: "probe-2681", pid: 3000, processStartTime: "444", ttyDevice: "555" }); const staleOwner = owner({ childToken: "stale-2681", pid: 4000, processStartTime: "666" }); + let removed = false; try { await acquireMemoryGuardClaims( stateDir, @@ -246,8 +247,11 @@ describe("memory guard owner claims", () => { const snapshot = await readMemoryGuardClaimsForTest(stateDir, probeOwner.sessionId); expect(snapshot.epoch).toBe(4); expect(snapshot.claims).toHaveLength(0); + await fs.rm(stateDir, { recursive: true }); + removed = true; + await expect(fs.access(stateDir)).rejects.toThrow(); } finally { - await fs.rm(stateDir, { recursive: true, force: true }); + if (!removed) await fs.rm(stateDir, { recursive: true, force: true }); } }); }); diff --git a/packages/coding-agent/test/gjc-runtime/team-runtime.test.ts b/packages/coding-agent/test/gjc-runtime/team-runtime.test.ts index 1300f31240..d7d0c2c73a 100644 --- a/packages/coding-agent/test/gjc-runtime/team-runtime.test.ts +++ b/packages/coding-agent/test/gjc-runtime/team-runtime.test.ts @@ -58,7 +58,10 @@ import { GjcTeamTaskStore, withGjcTeamTaskMutation, } from "../../src/gjc-runtime/team-store"; -import { workerMemoryGuardLedgerPath } from "../../src/gjc-runtime/team-worker-memory-guard"; +import { + readTeamWorkerMemoryGuardLedger, + workerMemoryGuardLedgerPath, +} from "../../src/gjc-runtime/team-worker-memory-guard"; import { gjcContinuationReservationDigest, isValidGjcContinuationAck, @@ -449,16 +452,10 @@ afterEach(async () => { clearPsmuxDetectionCache(); __setBinaryResolverForTests(null); __setExecutableIdentityResolverForTests(null); - for (const session of [ - "gjc-worktree-team", - "gjc-fail-team", - "gjc-split-fail-team", - "gjc-named-team", - "gjc-cleanup-team", - "gjc-dirty-cleanup-team", - ]) { - Bun.spawnSync(["tmux", "kill-session", "-t", session], { stdout: "ignore", stderr: "ignore" }); - } + // Team launches attach to an existing leader session; this suite creates no + // real tmux sessions. The tests that exercise tmux use per-test fake + // binaries, so probing a host tmux here is both unrelated and can hang on + // Windows when that runtime is unavailable or unresponsive. const roots = new Set(cleanupRoots); if (cleanupRoot) roots.add(cleanupRoot); cleanupRoots.clear(); @@ -3849,6 +3846,67 @@ describe("resolveGjcWorkerCommand invocation authority", () => { }); describe("team worker memory guard wiring", () => { + function expectRecord(value: unknown): asserts value is Record { + if (value === null || typeof value !== "object" || Array.isArray(value)) + throw new Error("Expected team API operation result to be a record"); + } + async function applyMemoryGuardWithStartupAck(input: { + teamName: string; + stateDir: string; + env: NodeJS.ProcessEnv; + token: string; + }): Promise> { + const before = Number.parseInt(await fs.readFile(path.join(cleanupRoot!, "tmux-split-count"), "utf8"), 10); + const config = await readTeamConfig(input.stateDir); + const oldPaneId = config.workers.find(worker => worker.id === "worker-1")?.pane_id; + expect(oldPaneId).toBeTruthy(); + const markerAbort = new AbortController(); + const successorAck = waitForFileText( + path.join(cleanupRoot!, "tmux-last-split"), + text => text.trim() === `${before + 1}\t${oldPaneId}\t%${before + 2}`, + markerAbort.signal, + ).then(() => + executeGjcTeamApiOperation( + "worker-startup-ack", + { + team_name: input.teamName, + worker_id: "worker-1", + protocol_version: "1", + replacement_token: input.token, + }, + cleanupRoot!, + input.env, + ), + ); + const replacement = executeGjcTeamApiOperation( + "apply-worker-memory-guard", + { + team_name: input.teamName, + worker_id: "worker-1", + platform: "linux", + automatic_action_allowed: true, + reason: "post-ack-injected-failure", + replacement_token: input.token, + }, + cleanupRoot!, + input.env, + ); + try { + const result = await replacement; + expectRecord(result); + if (process.platform !== "linux") { + markerAbort.abort(new Error("advisory host: no successor pane launch expected")); + await successorAck.catch(() => {}); + return result; + } + await successorAck; + return result; + } catch (error) { + markerAbort.abort(error); + await successorAck.catch(() => {}); + throw error; + } + } it("launches per-worker ledgers and publishes the worker ledger path to tmux commands", async () => { cleanupRoot = await createGitRepo(); const fakeTmux = await createFakeTmuxBin(cleanupRoot); @@ -4043,6 +4101,123 @@ describe("team worker memory guard wiring", () => { expect(task.claim?.owner).toBe("worker-2"); }, 15_000); + it("persists post-ACK publication failures in the retry budget with a stable incident", async () => { + cleanupRoot = await createGitRepo(); + const fakeTmux = await createFakeTmuxBin(cleanupRoot); + const env = { + GJC_SESSION_ID: TEST_SESSION_ID, + PATH: process.env.PATH ?? "", + GJC_TEAM_WORKER_COMMAND: "true", + GJC_TEAM_TMUX_COMMAND: fakeTmux, + GJC_TEAM_MEMORY_GUARD_STARTUP_TIMEOUT_MS: "5000", + }; + const snapshot = await startGjcTeam({ + workerCount: 1, + agentType: "executor", + task: "Post ACK retry ledger", + teamName: "memory-guard-post-ack-retry-team", + cwd: cleanupRoot, + platform: "linux", + env, + }); + const claim = await claimGjcTeamTask("memory-guard-post-ack-retry-team", "worker-1", cleanupRoot, env, "task-1"); + expect(claim.ok).toBe(true); + __setGjcTeamRuntimeTestSeamsForTests({ + memoryGuardHostPlatform: "linux", + memoryGuardReplacementPhase: phase => { + if (phase === "heartbeat_publication") throw new Error("injected_heartbeat_publication_failure"); + }, + }); + const first = (await applyMemoryGuardWithStartupAck({ + teamName: "memory-guard-post-ack-retry-team", + stateDir: snapshot.state_dir, + env, + token: "post-ack-retry-1", + })) as { result: string; ledger: { retry_count: number; last_incident_id?: string } }; + expect(first).toMatchObject({ result: "retrying", ledger: { retry_count: 1 } }); + expect(first.ledger.last_incident_id).toBeTruthy(); + const second = (await applyMemoryGuardWithStartupAck({ + teamName: "memory-guard-post-ack-retry-team", + stateDir: snapshot.state_dir, + env, + token: "post-ack-retry-2", + })) as { result: string; ledger: { retry_count: number; last_incident_id?: string } }; + expect(second).toMatchObject({ result: "blocked", ledger: { retry_count: 2 } }); + expect(second.ledger.last_incident_id).toBe(first.ledger.last_incident_id); + const persisted = (await Bun.file(workerMemoryGuardLedgerPath(snapshot.state_dir, "worker-1")).json()) as { + retry_count: number; + last_incident_id?: string; + }; + expect(persisted).toMatchObject({ retry_count: 2, last_incident_id: first.ledger.last_incident_id }); + const actions = await readTeamWorkerMemoryGuardLedger(path.join(snapshot.state_dir, "workers", "worker-1")); + expect(actions.slice(-2)).toMatchObject([ + { incident_id: first.ledger.last_incident_id, attempt: 1, result: "failed" }, + { incident_id: first.ledger.last_incident_id, attempt: 2, result: "blocked" }, + ]); + }, 30_000); + + it("blocks manual recovery when post-ACK rollback cannot prove successor termination", async () => { + cleanupRoot = await createGitRepo(); + const fakeTmux = await createFakeTmuxBin(cleanupRoot); + const env = { + GJC_SESSION_ID: TEST_SESSION_ID, + PATH: process.env.PATH ?? "", + GJC_TEAM_WORKER_COMMAND: "true", + GJC_TEAM_TMUX_COMMAND: fakeTmux, + GJC_TEAM_MEMORY_GUARD_STARTUP_TIMEOUT_MS: "5000", + }; + const snapshot = await startGjcTeam({ + workerCount: 1, + agentType: "executor", + task: "Ambiguous post ACK rollback", + teamName: "memory-guard-ambiguous-rollback-team", + cwd: cleanupRoot, + platform: "linux", + env, + }); + const claim = await claimGjcTeamTask( + "memory-guard-ambiguous-rollback-team", + "worker-1", + cleanupRoot, + env, + "task-1", + ); + expect(claim.ok).toBe(true); + __setGjcTeamRuntimeTestSeamsForTests({ + memoryGuardHostPlatform: "linux", + memoryGuardReplacementPhase: phase => { + if (phase === "heartbeat_publication" || phase === "rollback_successor_termination") + throw new Error(`injected_${phase}`); + }, + }); + const result = (await applyMemoryGuardWithStartupAck({ + teamName: "memory-guard-ambiguous-rollback-team", + stateDir: snapshot.state_dir, + env, + token: "ambiguous-rollback-1", + })) as { + result: string; + lifecycle_mutated: boolean; + ledger: { + state: string; + retry_count: number; + last_reason?: string; + last_replacement?: { rollback?: { successor_terminated: boolean; predecessor_state_restored: boolean } }; + }; + }; + expect(result).toMatchObject({ + result: "blocked", + lifecycle_mutated: true, + ledger: { + state: "blocked", + retry_count: 2, + last_replacement: { rollback: { successor_terminated: false, predecessor_state_restored: true } }, + }, + }); + expect(result.ledger.last_reason).toContain("rollback_failed:injected_rollback_successor_termination"); + const task = await readGjcTeamTask("memory-guard-ambiguous-rollback-team", "task-1", cleanupRoot, env); + expect(task.status).toBe("blocked"); + }, 30_000); it("caps Linux replacement retries and blocks the claimed task on the terminal failure", async () => { cleanupRoot = await createGitRepo(); const fakeTmux = await createFakeTmuxBin(cleanupRoot); diff --git a/packages/coding-agent/test/perf-corpus-rlm-analysis.test.ts b/packages/coding-agent/test/perf-corpus-rlm-analysis.test.ts index a645548617..2e5b4e381c 100644 --- a/packages/coding-agent/test/perf-corpus-rlm-analysis.test.ts +++ b/packages/coding-agent/test/perf-corpus-rlm-analysis.test.ts @@ -1517,7 +1517,7 @@ describe("trusted perf-corpus RLM analysis driver", () => { " handle.flush()", " os.fsync(handle.fileno())", " os.utime(report, ns=(original_stat.st_atime_ns, original_stat.st_mtime_ns))", - " if report.stat().st_ctime_ns == original_stat.st_ctime_ns:", + " if os.name != 'nt' and report.stat().st_ctime_ns == original_stat.st_ctime_ns:", " raise RuntimeError('in-place mutation did not update ctime_ns')", " return captured", "module._validate_sealed_inputs = capture_then_mutate_in_place", @@ -1558,6 +1558,66 @@ describe("trusted perf-corpus RLM analysis driver", () => { expect(result.evidenceStatus).toBe("INSUFFICIENT_EVIDENCE"); expect(validationCodes(result)).toContain("AUTHENTICATED_INPUT_METADATA_DRIFT"); }); + test("fails closed when an authenticated report disappears between rescan lstat and byte read", async () => { + const input = path.join(temporaryRoot, "rescan-read-disappearance-input"); + const output = path.join(temporaryRoot, "rescan-read-disappearance-output"); + await writeCorpus(input); + const sealedDigests = sealedDigestsByDirectory.get(path.resolve(input)); + if (!sealedDigests) throw new Error("sealed digests are unavailable for rescan/read disappearance test"); + const harness = [ + "import importlib.util, json, os, pathlib, sys", + "spec = importlib.util.spec_from_file_location('analysis', os.environ['DRIVER'])", + "module = importlib.util.module_from_spec(spec)", + "sys.modules['analysis'] = module", + "spec.loader.exec_module(module)", + "original_validate = module._validate_sealed_inputs", + "original_read = module._read_file_bytes", + "def capture_then_arm_rescan_removal(*args):", + " captured = original_validate(*args)", + " def remove_then_read(path, maximum_bytes):", + " if path.name == 'short-01.json':", + " path.unlink()", + " return original_read(path, maximum_bytes)", + " module._read_file_bytes = remove_then_read", + " return captured", + "module._validate_sealed_inputs = capture_then_arm_rescan_removal", + "result = module.run_analysis(", + " os.environ['INPUT'], os.environ['OUTPUT'], pathlib.Path(os.environ['PREREG']).read_bytes(),", + " os.environ['GIT_SHA'], os.environ['TREE_SHA'], os.environ['CLOSURE_DIGEST'],", + " os.environ['WORKTREE_FINGERPRINT'], os.environ['CONTROL_IDENTITY'], os.environ['CAPTURE_ID'],", + " os.environ['SCHEDULE_DIGEST'], os.environ['PROTOCOL_DIGEST'], os.environ['DRIVER_DIGEST'],", + " os.environ['PREREG_DIGEST'], os.environ['TEMPLATE_DIGEST'], os.environ['LEDGER_DIGEST'],", + " os.environ['MANIFEST_DIGEST'],", + ")", + "print(json.dumps(result['result']))", + ].join("\n"); + const subprocess = Bun.spawnSync(["python3", "-S", "-c", harness], { + env: { + ...process.env, + DRIVER: driverPath, + INPUT: input, + OUTPUT: output, + PREREG: preregistrationPath, + GIT_SHA: gitSha, + TREE_SHA: treeSha, + CLOSURE_DIGEST: expectedClosureDigest, + WORKTREE_FINGERPRINT: worktreeFingerprint, + CONTROL_IDENTITY: captureRuntimeControlIdentity, + CAPTURE_ID: captureId, + SCHEDULE_DIGEST: expectedScheduleDigest, + PROTOCOL_DIGEST: expectedProtocolDigest, + DRIVER_DIGEST: driverSha256, + PREREG_DIGEST: preregistrationSha256, + TEMPLATE_DIGEST: expectedTemplateSha256, + LEDGER_DIGEST: sealedDigests.attemptLedgerSha256, + MANIFEST_DIGEST: sealedDigests.rawManifestSha256, + }, + }); + expect(subprocess.exitCode).toBe(0); + const result = JSON.parse(decoder.decode(subprocess.stdout)) as ValidationResult; + expect(result.evidenceStatus).toBe("INSUFFICIENT_EVIDENCE"); + expect(validationCodes(result)).toContain("AUTHENTICATED_INPUT_METADATA_DRIFT"); + }); test.each([ ["report", "short-01.json"], ["attempt ledger", "perf-corpus-attempt-ledger.json"], diff --git a/packages/coding-agent/test/perf-corpus.test.ts b/packages/coding-agent/test/perf-corpus.test.ts index 07aa2c81cc..71aeec2ed5 100644 --- a/packages/coding-agent/test/perf-corpus.test.ts +++ b/packages/coding-agent/test/perf-corpus.test.ts @@ -1,4 +1,4 @@ -import { afterEach, beforeEach, describe, expect, test, vi } from "bun:test"; +import { afterEach, beforeEach, describe, expect, setDefaultTimeout, test, vi } from "bun:test"; import * as fs from "node:fs/promises"; import * as os from "node:os"; import * as path from "node:path"; @@ -11,6 +11,7 @@ import { normalizeProcessTreeRss, resolveGitProvenance, resolveMeasurementRuntimeProvenance, + spawnWithDistinctChildPid, updateMemoryObservedExtrema, } from "../bench/perf-corpus.bench"; import { @@ -34,6 +35,8 @@ import { validatePerfThresholdLedger, } from "../bench/perf-threshold.ledger"; +setDefaultTimeout(30_000); + const memoryControlKeys = [ "GJC_MEMORY_PROFILE", "GJC_MEMORY_ITERATIONS", @@ -125,23 +128,48 @@ describe("perf corpus schema + runner", () => { } expect(Number.isFinite(fixture.rssMemory.growthBytes)).toBe(true); } + }, 30_000); + test("retries only duplicate child PIDs and fails closed at the retry limit", () => { + const seenChildPids = new Set([41]); + const emittedPids = [41, 41, 42]; + const child = spawnWithDistinctChildPid(seenChildPids, () => ({ childPid: emittedPids.shift()! })); + expect(child.childPid).toBe(42); + expect(emittedPids).toEqual([]); + + let attempts = 0; + expect(() => + spawnWithDistinctChildPid( + seenChildPids, + () => { + attempts++; + return { childPid: 41 }; + }, + 3, + ), + ).toThrow("memory baseline child PID was reused after 3 attempts"); + expect(attempts).toBe(3); }); + test.each([ ["without runtime flags", []], ["with --smol", ["--smol"]], ["with --expose-gc", ["--expose-gc"]], ["with ordered runtime flags", ["--smol", "--expose-gc"]], - ] as const)("accepts the canonical direct invocation %s", (_name, execArguments) => { - const result = Bun.spawnSync([process.execPath, ...execArguments, canonicalBenchmarkPath], { - cwd: path.resolve(import.meta.dir, "../../.."), - env: { ...process.env, GJC_MEMORY_ITERATIONS: "1" }, - }); - expect(result.exitCode).toBe(0); - expect(new TextDecoder().decode(result.stderr)).toBe(""); - const report = JSON.parse(new TextDecoder().decode(result.stdout)) as PerfCorpusReport; - expect(report.runner.argv).toEqual(["bun", ...execArguments, logicalBenchmarkPath]); - expect(validatePerfCorpusReport(report)).toEqual({ ok: true, errors: [] }); - }); + ] as const)( + "accepts the canonical direct invocation %s", + (_name, execArguments) => { + const result = Bun.spawnSync([process.execPath, ...execArguments, canonicalBenchmarkPath], { + cwd: path.resolve(import.meta.dir, "../../.."), + env: { ...process.env, GJC_MEMORY_ITERATIONS: "1" }, + }); + expect(result.exitCode).toBe(0); + expect(new TextDecoder().decode(result.stderr)).toBe(""); + const report = JSON.parse(new TextDecoder().decode(result.stdout)) as PerfCorpusReport; + expect(report.runner.argv).toEqual(["bun", ...execArguments, logicalBenchmarkPath]); + expect(validatePerfCorpusReport(report)).toEqual({ ok: true, errors: [] }); + }, + 30_000, + ); test.each([ ["post-script flag", [canonicalBenchmarkPath, "--smol"]], @@ -159,28 +187,9 @@ describe("perf corpus schema + runner", () => { ); }); - test("rejects a dynamically imported wrapper that spoofs Bun.main and process.argv", async () => { - const wrapperDirectory = await fs.mkdtemp(path.join(os.tmpdir(), "gjc-perf-corpus-wrapper-")); - const wrapperPath = path.join(wrapperDirectory, "alternate-wrapper.ts"); - try { - await Bun.write( - wrapperPath, - `(Bun as { main: string }).main = ${JSON.stringify(canonicalBenchmarkPath)};\n` + - `process.argv.splice(0, process.argv.length, process.execPath, ${JSON.stringify(canonicalBenchmarkPath)});\n` + - `const { runPerfCorpusBenchmark } = await import(${JSON.stringify(canonicalBenchmarkPath)});\n` + - `process.stdout.write(JSON.stringify(runPerfCorpusBenchmark()));\n`, - ); - const result = Bun.spawnSync([process.execPath, wrapperPath], { - cwd: path.resolve(import.meta.dir, "../../.."), - }); - expect(result.exitCode).not.toBe(0); - expect(new TextDecoder().decode(result.stdout)).toBe(""); - expect(new TextDecoder().decode(result.stderr)).toContain( - "benchmark runner invocation is outside the frozen public contract", - ); - } finally { - await fs.rm(wrapperDirectory, { recursive: true, force: true }); - } + test("does not expose a programmatic runner that can bypass canonical argv authentication", async () => { + const benchmarkModule = await import(canonicalBenchmarkPath); + expect("runPerfCorpusBenchmark" in benchmarkModule).toBe(false); }); test("prefers checked-out HEAD over workflow SHA provenance", () => { const expectedSha = checkedOutHead(); @@ -496,7 +505,7 @@ describe("perf corpus schema + runner", () => { expect(baselines.map(baseline => baseline.surface)).toEqual([...REQUIRED_MEMORY_SURFACES]); expect(report.runner.memorySurfaceOrder).toEqual([...REQUIRED_MEMORY_SURFACES]); expect(report.runner.environment.GJC_MEMORY_SURFACE_ORDER).toBe(REQUIRED_MEMORY_SURFACES.join(",")); - }, 15_000); + }, 30_000); test("isolates each memory surface in a fresh Bun process using the preregistered order", () => { const customOrder = [...REQUIRED_MEMORY_SURFACES].reverse(); @@ -580,7 +589,7 @@ describe("perf corpus schema + runner", () => { expect(validatePerfCorpusReport(wrongOrdinal).errors).toContain( "memory baseline ordinal/surface identity must match runner.memorySurfaceOrder", ); - }, 15_000); + }, 30_000); test("rejects malformed explicit process-per-surface orders without normalization", () => { const canonicalOrder = REQUIRED_MEMORY_SURFACES.join(","); diff --git a/packages/coding-agent/test/session-resident-transition-seam.test.ts b/packages/coding-agent/test/session-resident-transition-seam.test.ts index e10aab9668..f8862c67dc 100644 --- a/packages/coding-agent/test/session-resident-transition-seam.test.ts +++ b/packages/coding-agent/test/session-resident-transition-seam.test.ts @@ -1147,7 +1147,7 @@ describe("resident-store transition seam", () => { } }); - it("T6 disposes a prepared candidate and retains staging state when managed publication collides", async () => { + it("T6 preserves the publication error when managed rollback cleanup fails", async () => { const root = makeTempDir("gjc-resident-transition-recovery-publish-"); const cwd = path.join(root, "workspace"); fs.mkdirSync(cwd, { recursive: true }); @@ -1158,30 +1158,34 @@ describe("resident-store transition seam", () => { if (!stagingFile) throw new Error("Expected staged recovery transcript"); const entriesBefore = JSON.stringify(staged.manager.getEntries()); let collisionPath: string | undefined; - const originalPublishNoReplace = ManagedSessionDescendantStore.prototype.publishNoReplace; - const publishNoReplace = vi - .spyOn(ManagedSessionDescendantStore.prototype, "publishNoReplace") - .mockImplementation(async function ( + const originalMoveExpectedNoReplace = ManagedSessionDescendantStore.prototype.moveExpectedNoReplace; + const moveExpectedNoReplace = vi + .spyOn(ManagedSessionDescendantStore.prototype, "moveExpectedNoReplace") + .mockImplementation(function ( this: ManagedSessionDescendantStore, - relativePath: string, - bytes: Uint8Array, + sourceRelativePath, + destinationRelativePath, + expected, ) { - if (relativePath.startsWith(".")) return await originalPublishNoReplace.call(this, relativePath, bytes); - collisionPath = path.join(this.dir, relativePath); - await originalPublishNoReplace.call(this, relativePath, Buffer.from("pre-existing collision")); - await originalPublishNoReplace.call(this, relativePath, bytes); + collisionPath = path.join(this.dir, destinationRelativePath); + this.publishNoReplaceSync(destinationRelativePath, Buffer.from("pre-existing collision")); + originalMoveExpectedNoReplace.call(this, sourceRelativePath, destinationRelativePath, expected); + }); + const originalRemoveExpected = ManagedSessionDescendantStore.prototype.removeExpected; + const removeExpected = vi + .spyOn(ManagedSessionDescendantStore.prototype, "removeExpected") + .mockImplementation(function (this: ManagedSessionDescendantStore, relativePath, expected) { + if (relativePath.includes(".recovery-staging")) throw new Error("rollback_cleanup_failed"); + return originalRemoveExpected.call(this, relativePath, expected); }); - const adopted = vi.spyOn(EphemeralBlobStore, "adoptVerifiedDir"); - const disposed = vi.spyOn(EphemeralBlobStore.prototype, "dispose"); try { await expect( staged.manager.promoteRecoveryHydrationAfterOwnershipReadyFence(staged.hydrationContext, { ownershipReady: true, }), ).rejects.toThrow("destination_conflict"); - expect(publishNoReplace).toHaveBeenCalledTimes(1); - expect(adopted).toHaveBeenCalledTimes(1); - expect(disposed).toHaveBeenCalledTimes(1); + expect(moveExpectedNoReplace).toHaveBeenCalledTimes(1); + expect(removeExpected).toHaveBeenCalledWith(expect.stringContaining(".recovery-staging"), expect.anything()); expect(collisionPath).toBeDefined(); expect(fs.existsSync(collisionPath!)).toBe(true); expect(staged.manager.getSessionFile()).toBe(stagingFile); @@ -1193,4 +1197,45 @@ describe("resident-store transition seam", () => { await staged.cleanup(); } }); + it("T6 leaves metadata-drifted managed staging outside exact rollback authority", async () => { + const root = makeTempDir("gjc-resident-transition-recovery-drift-"); + const cwd = path.join(root, "workspace"); + fs.mkdirSync(cwd, { recursive: true }); + const destination = SessionManager.managedDestination(cwd, getAgentDir()); + const staged = await stageMemoryGuardRecovery( + root, + destination, + `recovery drift predecessor ${"n".repeat(4096)}`, + ); + let stagingPath: string | undefined; + const originalMoveExpectedNoReplace = ManagedSessionDescendantStore.prototype.moveExpectedNoReplace; + vi.spyOn(ManagedSessionDescendantStore.prototype, "moveExpectedNoReplace").mockImplementation(function ( + this: ManagedSessionDescendantStore, + sourceRelativePath, + destinationRelativePath, + expected, + ) { + this.publishNoReplaceSync(destinationRelativePath, Buffer.from("pre-existing collision")); + try { + originalMoveExpectedNoReplace.call(this, sourceRelativePath, destinationRelativePath, expected); + } catch (error) { + stagingPath = path.join(this.dir, sourceRelativePath); + const drifted = new Date(Date.now() + 2_000); + fs.utimesSync(stagingPath, drifted, drifted); + throw error; + } + }); + const removeExpected = vi.spyOn(ManagedSessionDescendantStore.prototype, "removeExpected"); + try { + await expect( + staged.manager.promoteRecoveryHydrationAfterOwnershipReadyFence(staged.hydrationContext, { + ownershipReady: true, + }), + ).rejects.toThrow("destination_conflict"); + expect(stagingPath).toBeDefined(); + expect(removeExpected).not.toHaveBeenCalled(); + } finally { + await staged.cleanup(); + } + }); }); diff --git a/packages/coding-agent/test/session-storage.test.ts b/packages/coding-agent/test/session-storage.test.ts index b2c50bae88..f168e2f36d 100644 --- a/packages/coding-agent/test/session-storage.test.ts +++ b/packages/coding-agent/test/session-storage.test.ts @@ -501,6 +501,62 @@ describe("FileSessionStorageWriter certainty-aware close", () => { fs.closeSync(unrelatedFd); }); }); +describe("managed no-replace move", () => { + it("retains the exact staged source when the final name already exists", async () => { + const root = fs.realpathSync.native(fs.mkdtempSync(path.join(os.tmpdir(), "gjc-managed-move-collision-"))); + try { + const sessionDir = path.join(root, "session"); + const store = new ManagedSessionDescendantStore(managedDirectoryRoot(root), sessionDir); + await store.publishNoReplace(".successor.recovery-staging", Buffer.from("successor\n")); + await store.publishNoReplace("session.jsonl", Buffer.from("existing\n")); + const staged = store.readExpected(".successor.recovery-staging"); + if (!staged) throw new Error("Expected staged managed successor"); + + expect(() => store.moveExpectedNoReplace(".successor.recovery-staging", "session.jsonl", staged)).toThrow(); + expect(fs.readFileSync(path.join(sessionDir, ".successor.recovery-staging"), "utf8")).toBe("successor\n"); + expect(fs.readFileSync(path.join(sessionDir, "session.jsonl"), "utf8")).toBe("existing\n"); + } finally { + fs.rmSync(root, { recursive: true, force: true }); + } + }); + it.skipIf(process.platform === "linux")( + "rejects a substituted staged source before publishing an authority-absent move", + async () => { + const root = fs.realpathSync.native(fs.mkdtempSync(path.join(os.tmpdir(), "gjc-managed-move-source-swap-"))); + try { + const sessionDir = path.join(root, "session"); + const store = new ManagedSessionDescendantStore(managedDirectoryRoot(root), sessionDir); + await store.publishNoReplace(".successor.recovery-staging", Buffer.from("successor\n")); + const staged = store.readExpected(".successor.recovery-staging"); + if (!staged) throw new Error("Expected staged managed successor"); + + const source = path.join(sessionDir, ".successor.recovery-staging"); + const destination = path.join(sessionDir, "session.jsonl"); + const retained = path.join(sessionDir, "retained-original"); + const exactRestorePath = native.exactRestore; + const exactRestore = vi + .spyOn(native, "exactRestore") + .mockImplementation((sourcePath, destinationPath, identity) => { + fs.renameSync(sourcePath, retained); + fs.writeFileSync(sourcePath, "substituted\n"); + return exactRestorePath(sourcePath, destinationPath, identity); + }); + try { + expect(() => + store.moveExpectedNoReplace(".successor.recovery-staging", "session.jsonl", staged), + ).toThrow("identity_mismatch"); + expect(fs.existsSync(destination)).toBe(false); + expect(fs.readFileSync(source, "utf8")).toBe("substituted\n"); + expect(fs.readFileSync(retained, "utf8")).toBe("successor\n"); + } finally { + exactRestore.mockRestore(); + } + } finally { + fs.rmSync(root, { recursive: true, force: true }); + } + }, + ); +}); describe.skipIf(process.platform !== "darwin")("authority-absent managed replacement", () => { it("atomically replaces an existing file through the Darwin path", () => { diff --git a/packages/coding-agent/test/session/memory-guard-checkpoint.test.ts b/packages/coding-agent/test/session/memory-guard-checkpoint.test.ts index 03acf6c5aa..0524af80ba 100644 --- a/packages/coding-agent/test/session/memory-guard-checkpoint.test.ts +++ b/packages/coding-agent/test/session/memory-guard-checkpoint.test.ts @@ -1,4 +1,5 @@ -import { afterEach, describe, expect, it } from "bun:test"; +import { afterEach, describe, expect, it, vi } from "bun:test"; +import * as nodeFs from "node:fs"; import * as fs from "node:fs/promises"; import * as os from "node:os"; import * as path from "node:path"; @@ -18,6 +19,7 @@ import type { } from "@gajae-code/coding-agent/session/memory-guard-checkpoint-participant"; import { memoryGuardCanonicalJson } from "@gajae-code/coding-agent/session/memory-guard-checkpoint-participant"; import { SessionManager } from "@gajae-code/coding-agent/session/session-manager"; +import * as native from "@gajae-code/natives"; import { openRecoveryFsRoot, type RecoveryFsRoot } from "@gajae-code/natives"; const tempRoots: string[] = []; @@ -46,6 +48,29 @@ afterEach(async () => { for (const store of authStores.splice(0)) store.close(); for (const root of tempRoots.splice(0)) await fs.rm(root, { recursive: true, force: true }); }); +async function stageUnmanagedRecoveryPromotion(root: string) { + const checkpointRoot = path.join(root, "checkpoint-root"); + const restoreRoot = path.join(root, "restore-root"); + const sourceManager = await SessionManager.open(path.join(root, "sessions", "checkpoint.jsonl")); + sourceManager.appendMessage({ role: "user", content: "promotion recovery", timestamp: 0 }); + await sourceManager.flush(); + const lease = sourceManager.acquireMemoryGuardParticipantIngressLease(); + const checkpoint = await sourceManager.createMemoryGuardCheckpoint({ ingressLease: lease, checkpointRoot }); + lease.release(); + const authority = openRecoveryFsRoot(checkpointRoot); + const restored = await SessionManager.restoreMemoryGuardCheckpoint({ + incidentAuthority: authority, + participant: participantFromCheckpoint(checkpoint), + checkpoint, + destination: restoreRoot, + }); + if (restored.kind !== "staged") throw new Error("Expected staged recovery session"); + return { authority, restoreRoot, restored, sourceManager }; +} + +function recoveryStagingNames(directory: string): Promise { + return fs.readdir(directory).then(names => names.filter(name => name.includes(".recovery-staging"))); +} describe("memory guard checkpoint export/restore", () => { it("exports the closed checkpoint and restores a staged recovery session", async () => { @@ -163,6 +188,12 @@ describe("memory guard checkpoint export/restore", () => { expect(promotedTranscript).toBeDefined(); expect(path.basename(promotedTranscript!)).not.toStartWith("."); expect(await Bun.file(promotedTranscript!).exists()).toBe(true); + expect(await Bun.file(restored.transcriptIdentity.canonicalPath).exists()).toBe(false); + expect( + (await fs.readdir(path.dirname(promotedTranscript!))).some( + name => name.startsWith(`.${path.basename(promotedTranscript!)}.`) && name.endsWith(".recovery-staging"), + ), + ).toBe(false); await session.session.dispose(); await releaseMemoryGuardClaims(path.join(root, "claims"), claimsLease); } finally { @@ -170,6 +201,208 @@ describe("memory guard checkpoint export/restore", () => { await manager.close(); } }); + describe("recovery promotion failure atomicity", () => { + it("unmanaged final publish failure retains source and removes owned staging", async () => { + if (process.platform !== "linux") return; + const root = await makeTempRoot(); + const staged = await stageUnmanagedRecoveryPromotion(root); + const source = staged.restored.transcriptIdentity.canonicalPath; + const rename = vi.spyOn(native, "renameNoReplacePath").mockReturnValueOnce({ + ok: false, + code: "destination_exists", + mutationState: "not_committed", + durabilityState: "not_attempted", + reason: "destination_exists", + primitive: "renameat2_noreplace", + } as never); + try { + await expect( + staged.restored.manager.promoteRecoveryHydrationAfterOwnershipReadyFence( + staged.restored.hydrationContext, + { + ownershipReady: true, + }, + ), + ).rejects.toThrow(); + expect(await Bun.file(source).exists()).toBe(true); + expect(await recoveryStagingNames(staged.restoreRoot)).toEqual([]); + } finally { + rename.mockRestore(); + await staged.restored.cleanup(); + staged.authority.close(); + await staged.sourceManager.close(); + } + }); + + it("unmanaged published-byte verification failure retains source and no staging", async () => { + if (process.platform !== "linux") return; + const root = await makeTempRoot(); + const staged = await stageUnmanagedRecoveryPromotion(root); + const source = staged.restored.transcriptIdentity.canonicalPath; + const readFile = nodeFs.promises.readFile.bind(nodeFs.promises); + const spy = vi.spyOn(nodeFs.promises, "readFile").mockImplementation((async ( + file: string, + ...args: unknown[] + ) => { + if (!path.basename(file).startsWith(".") && String(file).endsWith(".jsonl")) return Buffer.from("tampered"); + return readFile(file, ...(args as [])); + }) as typeof fs.readFile); + try { + await expect( + staged.restored.manager.promoteRecoveryHydrationAfterOwnershipReadyFence( + staged.restored.hydrationContext, + { + ownershipReady: true, + }, + ), + ).rejects.toThrow(/Recovery successor final (?:identity|bytes) changed during promotion/); + expect(await Bun.file(source).exists()).toBe(true); + expect(await recoveryStagingNames(staged.restoreRoot)).toEqual([]); + } finally { + spy.mockRestore(); + await staged.restored.cleanup(); + staged.authority.close(); + await staged.sourceManager.close(); + } + }); + + it("unmanaged source retirement failure completes with verified duplicate and source", async () => { + if (process.platform !== "linux") return; + const root = await makeTempRoot(); + const staged = await stageUnmanagedRecoveryPromotion(root); + const source = staged.restored.transcriptIdentity.canonicalPath; + const exactUnlink = native.exactUnlink; + const unlink = vi + .spyOn(native, "exactUnlink") + .mockImplementation((pathname, identity) => + path.resolve(String(pathname)) === path.resolve(source) + ? ({ ok: false, code: "io_error" } as never) + : exactUnlink(pathname, identity), + ); + try { + await expect( + staged.restored.manager.promoteRecoveryHydrationAfterOwnershipReadyFence( + staged.restored.hydrationContext, + { + ownershipReady: true, + }, + ), + ).resolves.toBeUndefined(); + expect(await Bun.file(source).exists()).toBe(true); + expect( + (await fs.readdir(path.dirname(source))).some(name => !name.startsWith(".") && name.endsWith(".jsonl")), + ).toBe(true); + expect(staged.restored.manager.getSessionFile()).not.toBe(source); + unlink.mockRestore(); + await staged.restored.manager.close(); + expect(await Bun.file(source).exists()).toBe(false); + } finally { + unlink.mockRestore(); + await staged.restored.cleanup(); + staged.authority.close(); + await staged.sourceManager.close(); + } + }); + it("unmanaged retry adopts an identity-bound final left by ambiguous rollback", async () => { + if (process.platform !== "linux") return; + const root = await makeTempRoot(); + const staged = await stageUnmanagedRecoveryPromotion(root); + const source = staged.restored.transcriptIdentity.canonicalPath; + const realReadFile = nodeFs.promises.readFile.bind(nodeFs.promises); + const realExactUnlink = native.exactUnlink; + let finalPath: string | undefined; + const readFile = vi.spyOn(nodeFs.promises, "readFile").mockImplementation((async ( + file: string, + ...args: unknown[] + ) => { + if (!path.basename(file).startsWith(".") && String(file).endsWith(".jsonl")) { + finalPath = String(file); + return Buffer.from("tampered"); + } + return realReadFile(file, ...(args as [])); + }) as typeof fs.readFile); + const unlink = vi + .spyOn(native, "exactUnlink") + .mockImplementation((pathname, identity) => + finalPath && path.resolve(String(pathname)) === path.resolve(finalPath) + ? ({ ok: false, code: "io_error" } as never) + : realExactUnlink(pathname, identity), + ); + try { + await expect( + staged.restored.manager.promoteRecoveryHydrationAfterOwnershipReadyFence( + staged.restored.hydrationContext, + { ownershipReady: true }, + ), + ).rejects.toThrow(); + expect(await Bun.file(source).exists()).toBe(true); + expect(finalPath).toBeDefined(); + expect(await Bun.file(finalPath!).exists()).toBe(true); + readFile.mockRestore(); + unlink.mockRestore(); + + await staged.restored.manager.promoteRecoveryHydrationAfterOwnershipReadyFence( + staged.restored.hydrationContext, + { ownershipReady: true }, + ); + expect(await Bun.file(source).exists()).toBe(false); + expect(staged.restored.manager.getSessionFile()).toBe(finalPath); + } finally { + readFile.mockRestore(); + unlink.mockRestore(); + await staged.restored.cleanup(); + staged.authority.close(); + await staged.sourceManager.close(); + } + }); + it("unmanaged directory durability failure retains source after final publication", async () => { + if (process.platform !== "linux") return; + const root = await makeTempRoot(); + const staged = await stageUnmanagedRecoveryPromotion(root); + const source = staged.restored.transcriptIdentity.canonicalPath; + const realRename = native.renameNoReplacePath; + const realOpen = nodeFs.promises.open.bind(nodeFs.promises); + let promotionDirectory: string | undefined; + const rename = vi.spyOn(native, "renameNoReplacePath").mockImplementation((sourcePath, destinationPath) => { + promotionDirectory = path.dirname(destinationPath); + return realRename(sourcePath, destinationPath); + }); + const error = Object.assign(new Error("promotion directory sync failed"), { code: "EIO" }); + const open = vi.spyOn(nodeFs.promises, "open").mockImplementation((async ( + file: string, + ...args: unknown[] + ) => { + const handle = await (realOpen as (file: string, ...rest: unknown[]) => Promise)( + file, + ...args, + ); + if (promotionDirectory && path.resolve(file) === path.resolve(promotionDirectory)) { + (handle as unknown as { sync: () => Promise }).sync = async () => { + throw error; + }; + } + return handle; + }) as typeof fs.open); + try { + await expect( + staged.restored.manager.promoteRecoveryHydrationAfterOwnershipReadyFence( + staged.restored.hydrationContext, + { + ownershipReady: true, + }, + ), + ).rejects.toBe(error); + expect(await Bun.file(source).exists()).toBe(true); + expect(await recoveryStagingNames(staged.restoreRoot)).toEqual([]); + } finally { + open.mockRestore(); + rename.mockRestore(); + await staged.restored.cleanup(); + staged.authority.close(); + await staged.sourceManager.close(); + } + }); + }); it("fails closed when the retained transcript no longer matches the checkpoint descriptor", async () => { const root = await makeTempRoot(); @@ -232,3 +465,60 @@ describe("memory guard checkpoint export/restore", () => { ).resolves.toEqual({ kind: "blocked", reason: "checkpoint-mismatch" }); }); }); +describe("checkpoint directory durability", () => { + it("continues when Windows rejects directory sync with EPERM", async () => { + const root = await makeTempRoot(); + const checkpointRoot = path.join(root, "checkpoint-root"); + const manager = await SessionManager.open(path.join(root, "sessions", "checkpoint.jsonl")); + const participantRoot = path.join(checkpointRoot, "participants", manager.getSessionId()); + const platform = Object.getOwnPropertyDescriptor(process, "platform"); + const open = nodeFs.promises.open.bind(nodeFs.promises); + Object.defineProperty(process, "platform", { configurable: true, value: "win32" }); + const spy = vi.spyOn(nodeFs.promises, "open").mockImplementation((async (file: string, ...rest: unknown[]) => { + const handle = await (open as (file: string, ...args: unknown[]) => Promise)(file, ...rest); + if (path.resolve(file) === path.resolve(participantRoot)) + (handle as unknown as { sync: () => Promise }).sync = async () => { + throw Object.assign(new Error("EPERM"), { code: "EPERM" }); + }; + return handle; + }) as typeof fs.open); + try { + const lease = manager.acquireMemoryGuardParticipantIngressLease(); + const checkpoint = await manager.createMemoryGuardCheckpoint({ ingressLease: lease, checkpointRoot }); + lease.release(); + expect(await Bun.file(path.join(checkpointRoot, checkpoint.transcript.relative_path)).exists()).toBe(true); + } finally { + spy.mockRestore(); + if (platform) Object.defineProperty(process, "platform", platform); + await manager.close(); + } + }); + + it("fails closed for unexpected Windows directory sync errors", async () => { + const root = await makeTempRoot(); + const checkpointRoot = path.join(root, "checkpoint-root"); + const manager = await SessionManager.open(path.join(root, "sessions", "checkpoint.jsonl")); + const participantRoot = path.join(checkpointRoot, "participants", manager.getSessionId()); + const platform = Object.getOwnPropertyDescriptor(process, "platform"); + const open = nodeFs.promises.open.bind(nodeFs.promises); + const error = Object.assign(new Error("directory sync failed"), { code: "EIO" }); + Object.defineProperty(process, "platform", { configurable: true, value: "win32" }); + const spy = vi.spyOn(nodeFs.promises, "open").mockImplementation((async (file: string, ...rest: unknown[]) => { + const handle = await (open as (file: string, ...args: unknown[]) => Promise)(file, ...rest); + if (path.resolve(file) === path.resolve(participantRoot)) + (handle as unknown as { sync: () => Promise }).sync = async () => { + throw error; + }; + return handle; + }) as typeof fs.open); + try { + const lease = manager.acquireMemoryGuardParticipantIngressLease(); + await expect(manager.createMemoryGuardCheckpoint({ ingressLease: lease, checkpointRoot })).rejects.toBe(error); + lease.release(); + } finally { + spy.mockRestore(); + if (platform) Object.defineProperty(process, "platform", platform); + await manager.close(); + } + }); +}); diff --git a/packages/coding-agent/test/tools/resource-gc.test.ts b/packages/coding-agent/test/tools/resource-gc.test.ts index f1867e74cb..21a20c69ae 100644 --- a/packages/coding-agent/test/tools/resource-gc.test.ts +++ b/packages/coding-agent/test/tools/resource-gc.test.ts @@ -1267,5 +1267,9 @@ describe("resource GC settings precedence", () => { const settings = await Settings.init({ cwd: projectDir, agentDir }); expect(settings.get("browser.gc.idleMs")).toBe(222_222); + + resetSettingsForTest(); + expect(() => fs.rmSync(testDir, { recursive: true })).not.toThrow(); + expect(fs.existsSync(testDir)).toBe(false); }); });