Skip to content

fix(acp): share Hermes process across independent agent sessions - #3194

Closed
desmond-rai wants to merge 4 commits into
getpaseo:mainfrom
desmond-rai:feat/hermes-concurrency-guard
Closed

fix(acp): share Hermes process across independent agent sessions#3194
desmond-rai wants to merge 4 commits into
getpaseo:mainfrom
desmond-rai:feat/hermes-concurrency-guard

Conversation

@desmond-rai

@desmond-rai desmond-rai commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Linked issue

Closes #3193

Type of change

  • Bug fix
  • New feature
  • Enhancement
  • Refactor
  • Docs

Reasoning

Paseo's built-in Hermes integration currently launches one ACP subprocess per Paseo agent. That normally gives providers process isolation, but Hermes's durable profile is shared outside the workspace. Parallel agents therefore become concurrent OS-process writers to the same profile-level memory and configuration even when every agent has a separate Paseo workspace and Git worktree.

The user-visible workflow is ordinary Paseo parallelism: open Paseo, select Hermes, and start several agents. Users reasonably expect separate conversations, but they should not need to know that repository isolation does not isolate a provider's global durable state or that the built-in provider requires a one-process rule.

Hermes's ACP server already supports the primitive needed to fix this without merging conversations: one transport can host multiple native ACP sessions. This change makes that process sharing an explicit ACP-client capability and enables it only for the built-in hermes provider. Each Paseo agent still creates and owns a separate native ACP session, transcript, turn state, permissions, terminals, and cancellation path. The shared process only centralizes ownership of the profile writer.

The alternatives did not preserve the intended workflow:

  • Git worktrees: isolate repository files, not ~/.hermes or another provider home.
  • One profile clone per agent: the tested profile was about 24 GB, took about 45 seconds per clone, caused multi-agent startup timeouts, and fragmented one durable agent identity into divergent copies.
  • Serialize turns while keeping one process per agent: processes launch before startTurn(), so a turn gate cannot prevent concurrent profile writers. It also defeats parallel work.
  • Tell users to run one Hermes agent at a time: avoids the symptom by removing Paseo's parallel-agent workflow.

The implementation keeps ordinary ACP providers on the existing one-process-per-agent path. Process sharing is opt-in in ACPAgentClient; GenericACPAgentClient enables it only when providerId === "hermes".

What changes

  • Adds a provider/launch-environment-scoped shared ACP host and lease registry.
  • Uses one ClientSideConnection for many independent native ACP sessions.
  • Routes every callback by native ACP session ID and fails closed for missing, unknown, or stale IDs.
  • Removes only Paseo's per-agent variables (PASEO_AGENT_ID, PASEO_AGENT_CWD, PASEO_WORKSPACE_ID) from shared launch identity; other provider environment values remain part of the key and launch environment.
  • Makes catalog discovery, diagnostics, and importable-session probes reuse the same Hermes host rather than spawning probe writers.
  • Separates session lifetime from process lifetime: closing one session does not close sibling sessions.
  • Handles cancellation, session setup failures, provider exit, stale host replacement, daemon restart/resume, and idle shutdown.
  • Carries one timeout budget through host acquisition and ACP initialization.
  • Retains registry ownership until a timed-out process is confirmed exited, preventing an old and replacement Hermes process from overlapping.
  • Cancels pending permission requests when the shared process exits.

Goals

  • Parallel Hermes-backed Paseo agents use exactly one Hermes ACP OS process per launch identity/profile.
  • Every Paseo agent has a different native ACP session and independent turn history.
  • Native-session callback routing cannot leak events, permissions, terminals, or filesystem requests across agents.
  • Missing, stale, or unknown native session IDs fail closed.
  • Catalog and management probes cannot create a second profile writer.
  • Process death fails attached sessions and permits replacement only after the old process is known gone.
  • Existing non-Hermes ACP provider behavior is unchanged.
  • No profile selection, profile cloning, lock management, or workflow change is required from the user.

Non-goals

  • Sharing processes for all ACP providers. Other providers remain independent unless they explicitly opt in later.
  • Sharing conversation state between Paseo agents. Native ACP sessions remain separate.
  • Changing Hermes's memory policy or deciding which information Hermes persists.
  • Implementing a generic cross-process file lock around arbitrary provider homes.
  • Adding UI or protocol changes.

QA

Automated provider coverage

Commands run from the rebased feature worktree at 40b97e09893c7887cecc599af2da98c5053542c1:

$ npm exec vitest run -- src/server/agent/providers/acp-agent.test.ts src/server/agent/providers/generic-acp-agent.test.ts

Test Files  2 passed (2)
Tests       111 passed (111)

The tests cover:

  • one spawn with ten independent native sessions;
  • strict callback routing and unknown-session rejection;
  • per-session cancellation, close, permissions, terminals, and filesystem callbacks;
  • caller/provider environment preservation and per-agent environment exclusion;
  • environment mismatch rejection;
  • catalog request coalescing with per-caller deadlines;
  • catalog, diagnostic, and import probes reusing an active custom-environment host;
  • bounded ACP initialization and cleanup after hung handshakes;
  • process termination timeout ownership;
  • process exit, stale host replacement, and no overlapping replacement writer;
  • independent behavior for non-Hermes ACP providers.

Static and build checks

$ npm run lint
Found 0 warnings and 0 errors.
Finished on 3341 files.

$ npm run format:check
All matched files use the correct format.
Finished on 3577 files.

$ npm run generate:validators --workspace=@getpaseo/protocol
$ npm run typecheck --workspace=@getpaseo/client
$ npm run typecheck --workspace=@getpaseo/server
$ npm run typecheck --workspace=@getpaseo/app
$ npm run typecheck --workspace=@getpaseo/cli
# all exited 0 when run sequentially

$ npm run build:server
# server, client dependencies, and CLI exited 0

$ CSC_IDENTITY_AUTO_DISCOVERY=false npm run build:desktop -- --publish never --mac --arm64 --dir
# exited 0; packaged macOS app launched successfully after consistent local ad-hoc signing

The root typecheck script runs protocol generation and consumers concurrently. In this checkout that produced transient missing generated-message types. Running protocol generation first and each consumer sequentially passed. This PR does not touch protocol or generated files.

Full server-suite reconciliation

A highly parallel full server run completed 319 files and 4,701 tests, with 41 timeout/path failures concentrated in seven unrelated supervisor/workspace/WebSocket/Hub files:

Test Files  7 failed | 319 passed | 3 skipped
Tests       41 failed | 4701 passed | 43 skipped

Every affected area was then rerun serially with a canonical macOS temp directory:

$ TMPDIR=/private/tmp npm exec vitest run -- --maxWorkers=1 --no-file-parallelism \
    scripts/supervisor.logging.test.ts \
    src/server/websocket-server.browser-tools.test.ts \
    src/server/workspace-git-service.observation.test.ts \
    src/server/workspace-service-port-allocator.test.ts \
    src/server/hub/daemon-executions.test.ts \
    src/server/hub/execution-session.websocket.test.ts \
    src/server/hub/hub-cli-contract.test.ts \
    src/server/hub/relationship-controller.test.ts

Test Files  8 passed (8)
Tests       128 passed (128)

Real Hermes acceptance: source build

I ran ten real Paseo agents in parallel through matching server and CLI binaries, using Hermes 0.20.0 and deepseek:deepseek-v4-flash. A process-tree monitor started before provider discovery and sampled the daemon descendants throughout the run.

Paseo agents:                  10
Unique Paseo agent IDs:        10
Unique native ACP session IDs: 10
Idle terminal states:          10/10
Exact completion markers:      10/10
Process samples:               838
Maximum hermes acp processes:  1
Samples above one process:     0
Observed PID sets:             ['49683']

Real Hermes acceptance: packaged installed artifact

I built the macOS Electron package, installed that artifact, launched its bundled daemon and CLI on isolated disposable Paseo state, and repeated the ten-agent test against the packaged code rather than the source server.

Paseo agents:                  10
Unique Paseo agent IDs:        10
Unique native ACP session IDs: 10
Idle terminal states:          10/10
Exact completion markers:      10/10
Process samples:               681
Maximum hermes acp processes:  1
Samples above one process:     0
Observed PID sets:             ['43302']

Durable-memory behavior

Using a disposable Hermes profile:

  1. Two independent Paseo agents concurrently wrote different durable facts.
  2. Both writes completed and both facts appeared in the disposable profile.
  3. A third independent agent/session recalled both facts.
  4. All three agents had different native ACP session IDs.
  5. The process monitor never observed more than one hermes acp process.
Concurrent writers:            2
Independent reader:            1
Unique native ACP sessions:    3
Recall marker:                 MEMORY_RECALL_OK
Process samples:               418
Maximum hermes acp processes:  1
Samples above one process:     0

No acceptance marker was written to the user's normal Hermes profile.

Lifecycle and regression checks

  • Cancelling one session did not interrupt a concurrently running sibling session.
  • Killing the shared Hermes process failed both attached sessions.
  • The next agent created exactly one replacement process and completed.
  • Restarting the disposable Paseo daemon resumed a persisted agent and retained its earlier exchange.
  • A live non-Hermes mock provider agent completed normally on the independent-process path.
  • An independent read-only defect-first review of the final diff returned PASS after timeout, termination ownership, environment reuse, and pending-permission lifecycle findings were fixed and covered by tests.

Greptile's first public review identified that a timed-out management probe could invalidate a healthy shared host and fail unrelated active sessions. Commit 3d4653a70 now releases the abandoned probe lease without invalidating a host that has other references; an isolated host is still invalidated and replaced. A new regression test keeps an agent session attached while a catalog probe hangs and verifies that the process is not terminated. The same review flagged fixed-duration sleeps in timeout tests; all of those sleeps were replaced with explicit deferred synchronization. The updated focused suite is 111/111, and the full pre-commit lint, format, and workspace typecheck hook passes.

Platforms

Platform Tested Notes
Desktop macOS Yes Apple Silicon, macOS 15.6.1; source daemon and packaged Electron artifact
Desktop Windows No Process lifecycle is covered by mocked cross-platform unit tests; no Windows host was available
Desktop Linux No Process lifecycle is covered by mocked cross-platform unit tests; no Linux host was used
Web/iOS/Android Not affected No app UI or wire-protocol change

Checklist

  • One focused change
  • npm run typecheck passes when protocol generation and dependent workspaces run sequentially; see QA note about the root script's generation race
  • npm run lint passes
  • npm run format / npm run format:check passes
  • QA evidence
  • Tests added or updated where it made sense

@greptile-apps

greptile-apps Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR introduces daemon-scoped ACP process sharing for Hermes while retaining independent native sessions and the existing per-process behavior for other providers.

  • Shares one Hermes ACP host across provider-client replacements with per-session callback routing and leases.
  • Reuses the shared host for catalog, diagnostic, and import-session probes.
  • Adds timeout, process-exit, environment-identity, cancellation, and lifecycle regression coverage.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
packages/server/src/server/agent/providers/acp-agent.ts Implements shared ACP host acquisition, session routing, lease-based lifecycle management, bounded probes, and process replacement.
packages/server/src/server/agent/providers/acp-agent.test.ts Adds broad behavioral coverage for shared sessions, timeout cleanup, environment identity, routing, process exit, and deterministic lifecycle synchronization.
packages/server/src/server/agent/providers/generic-acp-agent.ts Enables shared-process behavior only for the built-in Hermes provider.
packages/server/src/server/agent/provider-registry.ts Threads the daemon-scoped Hermes process identity through built-in and derived provider construction.
packages/server/src/server/agent/provider-snapshot-manager.ts Owns a stable process-sharing scope across provider snapshot rebuilds.

Sequence Diagram

sequenceDiagram
  participant A1 as Paseo Agent 1
  participant A2 as Paseo Agent 2
  participant C as ACPAgentClient
  participant H as Shared Hermes Host
  A1->>C: createSession()
  C->>H: acquire lease
  C->>H: newSession()
  H-->>A1: native session 1
  A2->>C: createSession()
  C->>H: acquire lease
  C->>H: newSession()
  H-->>A2: native session 2
  A1->>C: close session 1
  C->>H: release lease 1
  Note over H: Host remains alive for session 2
Loading

Reviews (4): Last reviewed commit: "fix(acp): preserve shared host across pr..." | Re-trigger Greptile

Comment thread packages/server/src/server/agent/providers/acp-agent.ts Outdated
Comment thread packages/server/src/server/agent/providers/acp-agent.test.ts
@desmond-rai

Copy link
Copy Markdown
Contributor Author

Added a third commit (15ada19): fix(acp): buffer shared-router session updates received before registration.

While exercising this branch against a Hermes ACP agent that advertises slash commands via available_commands_update, the shared router dropped the notification: Hermes sends it immediately after the session/new response, before the client continuation registers the session, so the router threw 'unknown session' and the command batch (and early usage_update) was lost for the life of the session.

The router now buffers up to 100 session updates per unregistered session and replays them in arrival order from register(); unregister() and notifyProcessExit() discard pending buffers. Requests for unknown sessions (permission prompts, file/terminal IO) still throw.

Red-green regression test included ('delivers session updates that arrive before session registration'): fails on 3d4653a, passes with the fix. Full acp-agent + generic-acp-agent suites: 112 tests passed. Server typecheck clean.

Related upstream fix for the non-shared path (same race at the session level): #3324.

…ration

Agents may push session-scoped notifications (for example
available_commands_update) immediately after the session/new response,
before the client continuation registers the session with the shared
router. The router previously threw 'unknown session' and the daemon
dropped the notification, so Hermes skill slash commands never reached
the cached command list and the slash popup stayed empty.

Buffer up to 100 pending session updates per unregistered session and
replay them in arrival order from register(); unregister() and
notifyProcessExit() discard pending buffers. Requests for unknown
sessions (permission prompts, file and terminal IO) still throw.
@desmond-rai

Copy link
Copy Markdown
Contributor Author

Superseded by #3398. The shared-process approach serialized access but did not isolate durable Hermes memory between logical agents; #3398 replaces it with deterministic per-agent profiles and isolated HERMES_HOME.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: parallel Hermes agents launch concurrent writers against one profile

1 participant