Add progress updates for inbound A2A work

[dimavrem22]

Executive Summary

Adds immediate acknowledgement and periodic progress reporting for inbound A2A worker tasks.

• Defaults to one short progress update every three minutes, with a configurable interval.
• Persists acknowledgement and progress delivery so retries do not duplicate worker messages.
• Preserves cadence and runner ownership across recovery, follow-up, cancellation, and shutdown.

Description

The gateway now acknowledges an inbound A2A task before starting its Claude Code turn and owns a referenced progress runner for the lifetime of that worker turn. Progress copy comes from an isolated, tool-free, one-turn Claude session using the task text and at most eight normalized tool identifiers of 80 characters each. Tool arguments and results are never retained or included. Unsafe, terminal, empty, or tool-identifier-echoing model output uses one generic nonterminal fallback.

Acknowledgements and pending progress messages are persisted before delivery and reconciled only against exact worker-role task history. Referenced retries recover failed acknowledgement delivery, and every acknowledgement or terminal reply thread drains before cancellation or shutdown returns. Owner fencing prevents an older turn from stopping a follow-up runner, and persisted due times preserve the original cadence across restarts and follow-ups. Explicit terminal tools and automatic completion durably fence new progress and drain any in-flight send before replying; the same fenced caller message cannot rerun after an ambiguous reply outcome, while a genuine caller follow-up acquires a new runner. Every worker admission, duplicate reconciliation, acknowledgement retry, and recovery pass first verifies the authoritative task, context, active state, and latest caller message. Accepted work persists and runs only authoritative caller parts and metadata. Stale persisted generations are finalized without acknowledgement or model work, then recovery discovers the current submitted or working caller generation. This rejects delayed or spoofed generations even after an in-memory cancellation tombstone is lost on restart. Delayed webhooks for input-required, auth-required, and terminal tasks stop before persistence, progress, or model admission. Cancellation publishes a task-and-message generation tombstone before serializing with webhook admission, resolving the latest authoritative caller message when a cancellation omits its message ID. Current or blocked acknowledgement work cannot later start a model turn; reactivation requires a distinct caller-role message whose task, context, key, and active state all match the authoritative task. Shutdown similarly closes admission before it snapshots background work and returns retryable HTTP 503 responses for raced webhooks. Completion, cancellation, stopped server state, and gateway shutdown safely stop or drain background work. The default interval is 180 seconds through INKBOX_A2A_PROGRESS_INTERVAL_SECONDS.

Reason

Long-running A2A tasks previously appeared idle until their final response. Prompt pickup confirmation and occasional concise updates make active work visible without spending a full requester turn on each progress event.

Decisions

• Default cadence: Use three minutes to provide reassurance without making ordinary tasks noisy.
• Progress context: Let the isolated side model interpret bounded normalized tool identifiers instead of maintaining a keyword taxonomy or activity-specific canned messages.
• Requester behavior: Ignore submitted/working update events in the delegating Claude session because progress is informational task history, while actionable and terminal states still wake it.
• Delivery safety: Validate new, duplicate, retry, and recovered worker generations against authoritative caller state before persistence or delivery; use authoritative content rather than webhook content; persist pending text and caller input; reconcile exact worker-role history; drain blocking acknowledgement and terminal sends; serialize cancellation and shutdown with admission; and durably fence terminal replies so retries, follow-ups, and recovery remain idempotent.

Testing

• uv run --with pytest pytest -q --ignore=tests/contract: 543 passed, 23 skipped.
• uv run --with pytest pytest -q tests/contract: 5 passed against the installed Claude Agent SDK and Claude Code CLI.
• uv build: source distribution and wheel built successfully, including the progress module.
• Inbound A2A lifecycle tests: Expect caller text not to spoof reconciliation, failed acknowledgements to retain a referenced retry, pending progress to retry immediately after recovery, cadence to survive a near-boundary follow-up, and an older turn not to stop its replacement runner.
• Terminal tool race tests: Expect complete, ask-caller, and fail to fence before replying; an in-flight progress delivery must drain first, ambiguous reply failure must remain fenced, and a genuine caller follow-up must acquire a new runner.
• Automatic completion recovery: Expect completion to use the same durable fence; both committed and remote-still-working ambiguous outcomes must not rerun the same caller turn after restart, while a new caller message runs normally.
• Authoritative admission and stopped-state recovery: Expect task/context/message mismatches, non-caller latest messages, and inactive or stopped states to stop before persistence, acknowledgement, progress, or model work. A delayed canceled generation remains rejected after restart, while the authoritative active caller generation runs once and uses authoritative parts and caller metadata instead of spoofed webhook content.
• Duplicate, retry, and catch-up tests: Expect stale pending acknowledgements to stop without sending, spoofed duplicate payloads to be replaced with authoritative data, and background acknowledgement retries to reject stale generations. On restart, a persisted stale caller generation finalizes without running before the current authoritative caller generation runs once and its duplicate dedupes; worker acknowledgement and progress history never becomes caller work.
• Cancellation and shutdown tests: Expect worker and progress jobs to drain and release only their own lifecycle state; cancellation with or without a message ID and cancellation racing a blocked acknowledgement must suppress the current generation. Distinct spoofed, non-caller, wrong-context, and stopped follow-ups must remain fenced, while one authoritative active caller follow-up runs and its duplicate dedupes. Blocked acknowledgement and terminal replies must finish before stop returns, and webhooks racing cleanup must receive retryable HTTP 503 without scheduling new work.
• Live inbound-progress scenario: Expect a one-minute CI-only cadence, two ordered nonterminal summaries with at least one descriptive/non-fallback summary, and final result 4 + 6 = 10 with the unique marker.