feat(cluster-protocol): add the Rust WebSocket binding and cloud data-plane contract

[tomdps]

Parent

• [EPIC] Open Engine Cluster Protocol v1: programmatic interface and legacy Zeroshot bridge #643 — [EPIC] Open Engine Cluster Protocol v1: programmatic interface and legacy Zeroshot bridge

Execution

AFK hardening pass. Start only after required CI is green for the current dev SHA and the worktree is created from a freshly fetched origin/dev.

Outcome

Harden the Rust WebSocket request-cancellation lifecycle delivered by #796 so cancellation, fast completion, and same-ID reuse cannot retain or cross-delete per-connection request state.

Authoritative contracts/current state

#796 merged the complete WebSocket server/client binding and the target dev push CI passed at b83cfe98a64f99f2d9767c4de5dc92f02fbed572. Post-merge and superseded-PR review found three concrete races in crates/openengine-cluster-server/src/websocket.rs:

1. $/cancelRequest removes and aborts the task handle, but task abortion bypasses run_passthrough_request cleanup, leaving the request ID in ConnectionState::in_flight_ids for the connection lifetime.
2. spawn_passthrough spawns before inserting the abort handle. A fast request can finish and remove a not-yet-present registry entry, after which the caller inserts a stale completed handle.
3. The superseded feat: feat(cluster-protocol): add the Rust WebSocket binding and cloud data-plane contract #797 guard released in_flight_ids and removed cancel_registry as separate unowned operations. A same-ID retry could register between them and have its new cancellation handle deleted by the old request's guard.

The generic dispatcher, NDJSON binding, protocol wire contract, and client multiplexing are already complete. #797 is closed and unmerged; origin/dev remains the source of truth. This pass owns only the server-side lifecycle defects and focused regression proof.

Owned scope

• Make cancellation registration race-free: the cancellation capability must exist before the request task can complete or observe cancellation.
• Guarantee that every cancelled passthrough request releases both its in_flight_ids entry and cancellation-registry entry even when the task future is aborted or dropped.
• Guarantee that normal fast completion cannot leave a stale cancellation entry.
• Make cleanup ownership/generation-safe: an old request may remove only the exact cancellation registration and in-flight ownership it created, never a later same-ID request's state.
• Keep unknown/completed $/cancelRequest notifications as no-ops and preserve the existing no-rollback claim after commit.
• Add focused deterministic tests that hold a backend dispatch pending, cancel it, prove request-state cleanup and safe ID reuse, force the fast-completion registration schedule, and race old cleanup against a new same-ID request.
• Preserve the shared stdio::dispatch path and transport-neutral client multiplex implementation; do not duplicate either implementation.

Acceptance criteria

• A cancelled pending request cannot permanently reserve its JSON-RPC ID and leaves no cancellation handle after the cancellation settles.
• A completed request leaves neither an in-flight ID nor a cancellation handle, including the fastest completion schedule.
• Cancellation/completion races emit at most one terminal response for the original request and cannot corrupt a later request using the same ID.
• An old request's cleanup cannot delete, abort, or unlock a newer same-ID request's state.
• Cancellation and lifecycle bookkeeping remain bounded by active passthrough requests for the connection.
• Existing WebSocket, NDJSON, protocol-artifact, and generated-schema behavior remains unchanged.

Verification

cargo test -p openengine-cluster-server --test websocket
cargo test -p openengine-cluster-client --test websocket
cargo test -p openengine-cluster-testkit --test protocol_websocket
npm run rust:check
npm run protocol:check
npm run typecheck
npm run lint
npm run test:unit
opcore check --changed

Every named focused target must execute non-zero tests. Each new race regression must fail against merge commit b83cfe98a64f99f2d9767c4de5dc92f02fbed572 and pass after the repair. A sequential test that cannot schedule the race is not proof.

Non-goals

• Reimplementing the WebSocket binding, client, dispatcher, or protocol artifacts.
• Changing JSON-RPC cancellation semantics or claiming rollback of provider/tool side effects.
• TypeScript/npm client.
• Native daemon or hosted control plane.

Blocked by

• None; feat: feat(cluster-protocol): add the Rust WebSocket binding and cloud data-plane contract #796 and target-branch CI are complete.

Unblocks

• feat(cluster-protocol): publish the TypeScript cluster client with full v1 parity #748 — feat(cluster-protocol): publish the TypeScript cluster client with full v1 parity
• feat(zeroshot-rust): add authenticated local-daemon discovery and loopback host #692 — feat(zeroshot-rust): add authenticated local-daemon discovery and loopback host
• test(cluster-protocol): certify the legacy developer-preview matrix #655 — test(cluster-protocol): certify the legacy developer-preview matrix
• Orchestra #333 — Zeroshot cluster executor canary