fix(incremental): refresh stale scan inventory before batching

[jiang4wqy]

Summary

Incremental /understand runs previously trusted the retained scan inventory, allowing added files to be missed, deleted files to remain selectable, and rename or ignore-rule membership changes to escape graph pruning.

This PR reconciles retained inventory with validated current membership before incremental batching. Structural drift triggers a deterministic, atomic scan/import refresh; inventory deltas are propagated through effectiveChangedFiles and a retry-safe pending journal. The same validated membership now supplies bounded project context, while unsafe symlink or junction aliases fail closed.

Modified-only runs still avoid inventory refresh. If membership reconciliation and the pending journal leave no effective changes, batching exits before source export extraction or clustering and emits an empty-batch payload. Full mode remains outside the incremental refresh path.

Linked issue(s)

N/A

Root cause

compute-batches.mjs filtered Git changes against scan-result.json before synchronizing the retained inventory with the current repository. The Git changed-file list was therefore neither a complete analysis target set nor a sufficient old-graph pruning set.

What changed

• Compare retained inventory with exact current membership for every changed-files run, including an empty Git diff.
• Reuse the deterministic scanner and import extractor without invoking the scanner LLM or falling back to full analysis.
• Persist a strict pending-inventory-changes.json journal so additions and removals survive retries until Phase 7 cleanup.
• Build effectiveChangedFiles from validated Git paths, the before/after inventory symmetric difference, and pending journal paths.
• Require non-degraded scanner/import results and validate inventory records, import-map keys and targets, normalized paths, file types, and canonical containment.
• Validate .ua/legacy state directories and retained scan files before reading; atomically replace the scan and preserve the previous bytes on pre-commit failure.
• Derive bounded projectContext only from validated inventory membership in full and incremental flows, avoiding direct README/manifest re-reads and rejecting unsafe aliases.
• Preserve multi-line .understandignore rules as ordered individual patterns across LF, CRLF, and CR files; apply data-directory, root, and CLI rules in that order.
• Hard-exclude generated .ua/ and .understand-anything/ roots and fail closed on external symlink/junction aliases without disclosing external content.
• Support NUL-delimited Git paths and update the /understand empty-diff flow so hidden inventory drift can still produce incremental work.

Before / after

Validated against a repository-scale fixture with 8 additions, 2 deletions, 1 modification, and 1 rename:

Metric	Before	After
Inventory	453 stale files	459 current files
Valid current targets analyzed	1/10	10/10
Current targets missed	9	0
Deleted or rename-old paths selected	2	0
Unchanged files selected	—	0

The refreshed POSIX run, Windows CRLF/backslash run, and fresh full-scan oracle produced the same complete JSON document and SHA-256 digest (3ce73f778b8d001a548e29a18fef890aae097e6473d2f5955d417d474cb98e53). Ignore re-include/exclude, journal propagation, multi-line ignore ordering, and external-junction canaries also passed.

Scope

Relative to current main, the PR touches 12 files (10 modified, 2 added): +4607/-247 total. Runtime scripts account for +1815/-174, tests for +2690/-26, skill/agent documentation for +101/-46, and the CI workflow for +1/-1. It adds no runtime dependency or lockfile change.

Verification

• pnpm lint
• pnpm build
• Root suite: 29 files passed, 1 platform file skipped; 460 passed, 6 conditional skips
• Windows two-worker root configuration: three complete runs passed with 460/460 tests and no worker RPC timeout
• Core suite: 46 files, 963/963 passed
• Focused scanner/refresh/batching/import/security suite: 278/278 passed
• Python 3.12 suite: 94 passed, 1 case-insensitive-filesystem skip, 30 subtests passed
• Repository-scale incremental/full oracle and canaries: 32/32 assertions passed
• Performance/fast-path gate: 20/20 recorded commands, 57/57 assertions, and 20/20 independent checks passed
• Modified-only, empty-effective-change, and full-analysis paths verified with valid preconditions

On the 459-file fixture, direct refresh had a three-run median duration of 4.61 seconds and a median parent-process peak working set of 65.69 MiB.

Known verification boundaries

• A live Claude /understand LLM end-to-end run could not start because the local Claude CLI returned HTTP 403 before plugin startup. Scanner, import extraction, refresh, batching, oracle, build, and test surfaces were verified independently.
• The existing Louvain path uses Math.random(). Analysis targets, imports, and exported symbols were stable; a test-only seeded RNG was used only for byte-level community-number parity across path formats.
• Cross-platform lstat/realpath checks fail closed before reads, but Node's path APIs cannot completely remove the syscall-level race between validation and open without a platform-specific no-follow file-handle implementation.
• Modified-only mode intentionally retains the existing import-map fast path. Recomputing imports for every modified source file is a separate behavior change outside this PR.

Versioning

• Version is 2.9.4 in all six manifests required by CLAUDE.md.