Windows: execenv hermes-home mirror fails intermittently with state.db-shm lock violation

[clearrouteit-bot]

Bug report: Multica agent tasks fail intermittently — "hermes-home" mirror hits a Windows file lock on state.db-shm

Summary

Multica agent tasks intermittently fail during execution-environment setup with:

prepare execution environment: execenv: prepare hermes-home: mirror shared hermes home:
mirror state.db-shm: copy <HERMES_HOME>\state.db-shm → <workspace>\hermes-home\state.db-shm:
read <HERMES_HOME>\state.db-shm: The process cannot access the file because another
process has locked a portion of the file.

This happens whenever the Hermes desktop app is running at the same time a Multica agent
task starts (which for most users is effectively always, since Hermes runs continuously in
the background/tray).

Environment

• OS: Windows 11 Pro (build 10.0.26200)
• Multica desktop: @multicadesktop\Multica.exe, daemon binary resources\bin\multica.exe
• Hermes: hermes-agent (venv-based background agent) + Electron desktop app
• Shared Hermes home: %LOCALAPPDATA%\hermes\ (contains state.db, state.db-wal, state.db-shm)

Root cause

During execenv.Prepare (source path embedded in the daemon binary as
github.com/multica-ai/multica/server/internal/daemon/execenv/hermes_home.go), Multica
mirrors the shared Hermes home directory into each per-task workspace via
prepareHermesHome → mirrorSharedHermesHome, using a plain OS-level file copy.

state.db is a SQLite database opened in WAL mode. On Windows, SQLite's WAL mode takes
byte-range locks on the companion state.db-shm (shared-memory) file for as long as any
connection is open. Windows enforces those locks against any other process trying to open
the file — including a simple read for copying — so the copy fails with
ERROR_LOCK_VIOLATION ("locked a portion of the file") whenever Hermes has an active
connection to its database, which is continuously while Hermes is running.

This is not a rare race: our daemon log (~/.multica/profiles/<profile>/daemon.log) shows
7+ task failures with this exact error over a few hours, interleaved with tasks that
succeeded (when the copy happened to land between WAL checkpoints). On failure, the code
takes one fallback path ("execenv: refresh hermes-home failed; forcing fresh prepare")
but that fresh prepare hits the same lock and the task is failed outright — there is no
retry/backoff on the copy itself.

Reproduction

1. Ensure the Hermes desktop app is running normally (background hermes.exe process(es)
   alive, actively using %LOCALAPPDATA%\hermes\state.db).
2. Send a message to any Multica agent that triggers a new task / workspace prepare.
3. Repeat several times in a short window. A meaningful fraction of tasks fail with the
   error above.

Confirmed independently: attempting File.Open(state.db-shm, Read, None) from an external
process while Hermes is running fails immediately with the same OS lock error — this isn't
Multica-specific, it's inherent to copying a live SQLite WAL -shm file on Windows.

Suggested fixes (any of these would resolve it)

1. Don't raw-copy state.db-shm/state.db-wal at all. These are regenerated
   automatically by SQLite the next time any connection opens state.db. Only state.db
   itself needs to exist in the mirrored home; skipping the -shm/-wal files entirely
   removes the lock hazard.
2. Use a SQLite-safe backup (e.g. the SQLite Online Backup API, or VACUUM INTO) instead
   of an OS-level file copy, which is lock-aware and safe to run against a live database.
3. Retry with backoff on this specific copy failure — the log shows plenty of
   nearby-in-time successful execenv: prepared env calls, so a short retry (e.g. 3 attempts,
   200-500ms apart) would very likely succeed without any deeper redesign.

Impact

Agent tasks fail non-deterministically whenever the user has Hermes open, with no
workaround exposed in Multica's settings/config (~/.multica/config.json and the daemon's
env vars were checked — no flag to disable or redirect hermes-home mirroring exists today).