Skip to content

docs: propose session-independent execution correlation (#565) - #574

Merged
imran-siddique merged 2 commits into
agentrust-io:mainfrom
Yatsuiii:feat/565-execution-correlation
Aug 26, 2026
Merged

docs: propose session-independent execution correlation (#565)#574
imran-siddique merged 2 commits into
agentrust-io:mainfrom
Yatsuiii:feat/565-execution-correlation

Conversation

@Yatsuiii

@Yatsuiii Yatsuiii commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

What

Records the agreed design for #565 in docs/spec/execution-correlation.md.

The document keeps the current identity finding prominent. initialize creates no client negotiated identity. session_id and call_id are gateway generated. workflow_id is supplied by the caller but is not a validated execution identity.

Settled decisions

  1. execution_id is a typed AuditEntry field, not a detail key.
  2. Collision within one authenticated agent identity is refused.
  3. The TRACE Claim does not enumerate execution_id values.
  4. Reuse after a terminal outcome is refused with no replay window.
  5. execution_id and workflow_id remain independent.

Claim boundary

The document records the boundary raised in trace-spec#66:

attempt identity ≠ logical operation identity ≠ external outcome identity

A shared execution_id establishes correlation only. Treating two requests as the same logical operation also requires an immutable canonical action or intent binding. A different binding under the same identifier is a collision or mutated operation and must be refused. This change does not invent another schema field for that binding.

Correctness requirements

Collision and replay enforcement requires authoritative execution state keyed by authenticated agent identity and execution_id. Reservation must occur atomically before upstream invocation, and terminal state must be durable before later replay classification.

Scope

Documentation only. No running behavior or schema changes are made in this PR.

Verification

  1. git diff --check passes.
  2. Ruff passes for src/cmcp_runtime, src/cmcp_verify, and tests.
  3. MyPy passes for 64 source files.
  4. All cited source locations were checked against the current branch.
  5. The complete local test command stalled in gateway conformance and was stopped. GitHub CI remains required for the final verdict.

DCO sign off

I certify that I wrote or have the right to submit this contribution and agree to the Developer Certificate of Origin.

@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

…#565)

Draft proposal, not an adopted spec, opened so the shape can be argued
before anything is built.

Starts by writing down what cMCP evidence actually binds on today, since
the dependence is not where "MCP session" suggests. `initialize` mints
nothing, it negotiates a protocol version and returns capabilities.
Everything joins on `session_id`, a gateway-side uuid4 from
SessionManager.create_session, with one hardware-anchored chain root and
one TRACE Claim issued at close.

The gap is that `session_id` carries two jobs that stop being compatible
once work arrives as independent requests: it scopes the evidence bundle
and it is the only thing relating two calls to one intent. One long-lived
session makes the claim unbounded and late. A session per request makes
multi-request work unjoinable. The roadmap requires it stay joinable.

Proposes a caller-asserted, gateway-validated `execution_id` carried in
`params._cmcp` beside the existing `workflow_id`, sitting alongside
`session_id` rather than replacing it, so sessions keep scoping bundles
and correlation crosses them.

Answers agentrust-io#565's four acceptance-evidence items explicitly, including the
one it cannot answer: protocol-version vectors belong with the P1 SDK
conformance matrix rather than being invented here.

Leaves five decisions open for a maintainer rather than resolving them,
including whether the identifier is an AuditEntry field or lives in
`detail`, and how collision inside one agent identity should behave.

Every claim about current behaviour cites a file and line verified
against origin/main.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KDXJ4ghkW6v56W8St2w5kg
Signed-off-by: Yatsuiii <battyrises@gmail.com>
@imran-siddique

Copy link
Copy Markdown
Member

Ruling on all five. You were right not to settle them alone, and right that a wrong shape here propagates into #566, #567 and #568.

First, the finding, because it is the part that stands regardless of what happens to the proposal. Verified at pinned 21df5b42: initialize at server.py:449 negotiates a protocol revision and mints nothing, and manager.py:118 is session_id = str(uuid4()), gateway-side. So "MCP session" suggests a client-negotiated identity that does not exist. That is the same discovery your #571 thread reached from the other direction, where call_id is also a fresh server-side uuid4. Three identifiers, none of them correlatable by the caller. Write that down in the document even if everything below changes.

1. Field on AuditEntry, not detail

A field. The schema change is the smaller cost.

detail is where things go when we have not decided what they are. Something the verifier must key on, and that #567 and #568 will both consume, is not that. Putting a correlation identifier in a free-form dict means every consumer re-derives its own expectations about presence, type and absence, and they will not agree.

The append-only hashed structure argues for a field rather than against one: a typed field that is always present, even as null, hashes predictably, while a key that appears in detail only sometimes changes the shape of what is hashed depending on a value's presence.

2. Collision: refuse

Refuse, inside a single agent identity.

A correlation identifier whose collisions are recorded rather than refused is not an identifier, it is a hint, and everything built on it inherits the ambiguity. #568 in particular cannot do anything useful with "these two executions might be the same one".

Refusing is also the honest failure: a caller that reuses an execution_id has a bug or is an attacker, and both should be told immediately rather than discovered by an auditor later.

3. No, the TRACE Claim does not enumerate them

An enumeration is unbounded, grows with session length, and duplicates what the audit chain already orders. The Claim binds the chain; the chain carries the identifiers.

There is also a disclosure argument: a Claim is the artifact most likely to be shared outside the operator, and a list of execution identifiers leaks work volume and shape to a reader entitled only to verify one record.

4. "Refuse reuse after a terminal outcome" is enough. No replay window.

A window is a clock, and a clock is a second source of truth that has to be synchronised, configured and reasoned about at verification time years later. We have just spent a week on what happens when a verifier's clock disagrees with an issuer's.

Terminal outcome is already the boundary this system understands. #571 landed not_attempted and outcome_unknown precisely to make that boundary explicit, so key the refusal on it and inherit the work rather than introducing a parallel notion of expiry.

One consequence to write down: an execution that never reaches a terminal outcome is never reusable and never expires. That is the fail-closed direction and it is correct, but it means a stuck execution holds its identifier permanently, and someone should know that before they see it.

5. Independent of workflow_id

They answer different questions. workflow_id groups work that belongs together by intent; execution_id identifies one unit of work for correlation across requests. Deriving either from the other collapses two questions into one and the collapse is not reversible.

Independent, with the relationship documented rather than encoded. If it turns out every execution_id in practice has exactly one workflow_id, that is an observation worth recording, not a constraint worth enforcing.

What I would like

Fold these into the document as decided rather than open, keep the initialize/uuid4 finding prominent, and mark it agreed rather than proposed. Then it is the thing #566, #567 and #568 build against.

Merging once that lands. Thanks for opening it for argument instead of implementing a guess.

Signed-off-by: Yatsuiii <battyrises@gmail.com>

@imran-siddique imran-siddique left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All five landed, and you carried the consequences rather than just the verdicts. That is the difference between a document that records a decision and one that can be built against.

Two places where that shows.

On the replay window. I said key the refusal on terminal outcome and inherit #571's work. You wrote:

This rule includes outcome_unknown: uncertainty must not become permission to repeat an irreversible effect.

That sentence is doing more than the rule I gave you. outcome_unknown is exactly the state where a retry layer is most tempted to try again, and saying plainly that uncertainty is not permission closes the reading where someone treats "we do not know" as "so it probably did not happen".

On the cost of that choice. I asked you to write down that a stuck execution holds its identifier forever, and you did, including the part I had not said:

A stuck execution therefore holds its identifier until an explicit recovery design exists. This is the fail closed result.

Naming that a recovery design does not exist yet, rather than leaving the reader to discover it, is what stops this being rediscovered as a bug in six months.

Two additions I did not ask for and would have wanted.

Scoping collision so the same value under different agent identities does not collide. My ruling said refuse within one agent identity and I did not say what happens across identities; leaving that implicit would have produced a verifier that refuses legitimate values from unrelated agents.

And a missing execution_id recording null in the typed field rather than synthesizing one, which gives consumers the property that matters: "a present value came from the validated caller path." A synthesized identifier would have looked identical to a real one, which is the same class of defect as the reported-but-unenforced fields we have been fixing all week.

Keeping the reasoning on workflow_id rather than only the answer is right too. "If operational evidence later shows a one to one relationship, that is an observation, not a constraint" is the sentence that stops someone encoding the correlation the first time they notice it.

Docs only, real checks green. Merging, and this is now the shape #566, #567 and #568 build against.

Thanks for opening it as a proposal and arguing the shape first. The five questions were the right five, and I would rather answer those than review an implementation of a guess.

@imran-siddique
imran-siddique merged commit e7c2956 into agentrust-io:main Aug 26, 2026
10 of 11 checks passed
Yatsuiii added a commit to Yatsuiii/cmcp that referenced this pull request Sep 3, 2026
…st-io#565)

First implementation slice for session-independent execution correlation,
following the design merged in agentrust-io#574 (docs/spec/execution-correlation.md).

ExecutionRegistry is the authoritative execution-lifecycle record: a
SQLite-backed deep module, one instance per process, keyed by
(agent_identity, execution_id) with a PRIMARY KEY on the pair.

- admit() runs under BEGIN IMMEDIATE and either reserves a fresh in_flight
  row or classifies an existing key as a replay (in_flight / terminal /
  outcome_unknown) or a collision (changed action binding). It never
  rewrites a row and never returns "proceed" for one.
- finalize() moves in_flight to completed (transport delivered a response)
  or outcome_unknown (anything earlier). Both terminal, neither replayable.
  A second call is a no-op.
- recover() runs once at startup and seals every still-in_flight row as
  outcome_unknown, so a crash between admit and finalize can never admit
  another invocation.

The action binding reaches admit() as an opaque digest string; the registry
only stores and byte-compares it. Its canonical construction is issue agentrust-io#588's.
execution/binding.py carries provisional_action_binding, an explicit stub
with an undecided preimage that reuses the existing RFC 8785/JCS
canonicalizer (cmcp_runtime.catalog.approval) so no second serializer is
introduced. valid_execution_id bounds the identifier to 1-200 printable
non-space ASCII characters before it can reach the durable key.

Signed-off-by: Yatsuiii <battyrises@gmail.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Yatsuiii added a commit to Yatsuiii/cmcp that referenced this pull request Sep 3, 2026
…#565)

execution_id is a typed AuditEntry field, always serialized (null when the
caller asserted none), never a detail key. A present value came through the
validated admission path in ExecutionRegistry; null means no assertion was
made and none was synthesized. AuditChain.append gains the matching keyword.

Per the agentrust-io#574 design: the TRACE Claim does not enumerate execution_id values;
the audit entry carries the join key so bundles stay joinable offline.

Signed-off-by: Yatsuiii <battyrises@gmail.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
imran-siddique added a commit that referenced this pull request Sep 8, 2026
)

* feat(execution): add durable ExecutionRegistry state machine (#565)

First implementation slice for session-independent execution correlation,
following the design merged in #574 (docs/spec/execution-correlation.md).

ExecutionRegistry is the authoritative execution-lifecycle record: a
SQLite-backed deep module, one instance per process, keyed by
(agent_identity, execution_id) with a PRIMARY KEY on the pair.

- admit() runs under BEGIN IMMEDIATE and either reserves a fresh in_flight
  row or classifies an existing key as a replay (in_flight / terminal /
  outcome_unknown) or a collision (changed action binding). It never
  rewrites a row and never returns "proceed" for one.
- finalize() moves in_flight to completed (transport delivered a response)
  or outcome_unknown (anything earlier). Both terminal, neither replayable.
  A second call is a no-op.
- recover() runs once at startup and seals every still-in_flight row as
  outcome_unknown, so a crash between admit and finalize can never admit
  another invocation.

The action binding reaches admit() as an opaque digest string; the registry
only stores and byte-compares it. Its canonical construction is issue #588's.
execution/binding.py carries provisional_action_binding, an explicit stub
with an undecided preimage that reuses the existing RFC 8785/JCS
canonicalizer (cmcp_runtime.catalog.approval) so no second serializer is
introduced. valid_execution_id bounds the identifier to 1-200 printable
non-space ASCII characters before it can reach the durable key.

Signed-off-by: Yatsuiii <battyrises@gmail.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* feat(audit): add typed execution_id field to AuditEntry (#565)

execution_id is a typed AuditEntry field, always serialized (null when the
caller asserted none), never a detail key. A present value came through the
validated admission path in ExecutionRegistry; null means no assertion was
made and none was synthesized. AuditChain.append gains the matching keyword.

Per the #574 design: the TRACE Claim does not enumerate execution_id values;
the audit entry carries the join key so bundles stay joinable offline.

Signed-off-by: Yatsuiii <battyrises@gmail.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* feat(proxy): reserve execution identity before upstream invocation (#565)

Wire ExecutionRegistry into the tool call path.

- server.py reads _cmcp.execution_id beside workflow_id; the two stay
  independent. A non-string is treated as absent.
- proxy._admit_execution reserves the execution identity at step 3a.5,
  immediately before upstream invocation. execution_id is bound-checked
  first (valid_execution_id); a malformed value is denied
  (execution_invalid_execution_id) with no reservation and is not written
  to the audit entry. The action binding comes from an injected
  action_binding_fn, so this module never encodes the #588 preimage.
- A replay or collision is refused here and never reaches upstream. The
  refusal is audited under the asserted execution_id.
- finalize() is called from the single terminal audit write
  (_append_call_terminal), so replay and collision policy is not spread
  across handlers. completed only when the transport delivered a response;
  anything earlier is outcome_unknown.
- cli.build_server constructs one process-wide registry, runs recover()
  before the gateway serves traffic, and injects provisional_action_binding.

A terminal transition is durable before a later request is classified as
replay, because finalize() commits its own transaction before any later
admit() runs. The terminal audit entry and the execution row are in
separate SQLite databases and do not share one transaction; a crash in the
gap leaves the row in_flight for recover() to seal. See
docs/spec/execution-correlation.md "Known limitations".

Signed-off-by: Yatsuiii <battyrises@gmail.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* docs(spec): record the #565 execution-correlation implementation slice

Add an "Implementation (first slice)" section to
docs/spec/execution-correlation.md describing what this slice builds:
ExecutionRegistry as the authoritative state owner, the admit / finalize /
recover contract, the opaque-digest binding seam, and execution_id ingress
validation.

The "Known limitations" subsection records that the terminal audit entry and
the execution row are in separate SQLite databases with no shared
transaction, that there is no cross-process reservation fencing beyond
SQLite BEGIN IMMEDIATE plus busy_timeout, and that this slice reports a
shared asserted execution_id without claiming exactly-once external
execution.

The action-binding construction (preimage, JCS member ordering, digest
representation) stays deferred to #588; this doc only describes what the
slice consumes.

Signed-off-by: Yatsuiii <battyrises@gmail.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(execution): drop the unreachable insert-race handler and cover the rollback paths

Codecov flagged nine uncovered lines in ExecutionRegistry: the recover()
rollback path and an admit() sqlite3.IntegrityError handler for a lost insert
race.

The IntegrityError handler was unreachable. Verified against two independent
connections on one database file: the second BEGIN IMMEDIATE blocks on the
first connection's RESERVED lock, and once the first commits, the second's
SELECT runs inside its own transaction and observes the committed row, so it
classifies rather than reaching the INSERT. When the lock is held past
busy_timeout the caller gets sqlite3.OperationalError, not IntegrityError.
Within one process the registry's own threading.Lock serialises. No path
produces the exception the handler caught, so it is removed rather than tested.

The spec said "a lost insert race re-reads and classifies", which described a
path that cannot occur. It now says a competing writer either observes the
committed reservation or fails closed if the lock timeout is exceeded.

Tests added for the paths that are reachable:

- admission rollback under an injected persistence failure, asserting no row
  is left behind and that a later admit still succeeds;
- recovery rollback under an injected persistence failure, asserting the row
  stays in_flight for a later recover() rather than half-sealed;
- two independent ExecutionRegistry instances on one file admitting the same
  key simultaneously, asserting exactly one ADMITTED and one REPLAY_IN_FLIGHT.

Execution package coverage 92% to 100%.

Signed-off-by: Yatsuiii <battyrises@gmail.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: refuse present non-string execution IDs at HTTP ingress

Signed-off-by: Imran Siddique <imran.siddique@opaque.co>

* fix(execution): keep incomplete correlation unavailable

---------

Signed-off-by: Yatsuiii <battyrises@gmail.com>
Signed-off-by: Imran Siddique <imran.siddique@opaque.co>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Imran Siddique <imran.siddique@opaque.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants