Skip to content

feat: Nexus Service support (workflow-backed Operations) - #9

Merged
tyler5673 merged 5 commits into
mainfrom
feat/nexus-service
Sep 3, 2026
Merged

feat: Nexus Service support (workflow-backed Operations)#9
tyler5673 merged 5 commits into
mainfrom
feat/nexus-service

Conversation

@tyler5673

@tyler5673 tyler5673 commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Summary

Linear: DX-745

Nexus Service support for Temporal Nexus, exposing the six You.com Activities as cross-Namespace Operations on top of the existing Activity layer. Callers in other Namespaces reach You.com through a Nexus Endpoint and a durable, typed contract.

Activity-only users are unaffected — youdotcom_temporal.nexus is a separate import.

Why every Operation is async (workflow-backed)

Nexus synchronous Operations must finish within the 10-second handler deadline. Several You.com calls routinely exceed it:

  • search with full-page extraction (crawl_timeout up to 60s)
  • contents with multiple URLs and a high per-URL crawl_timeout
  • research and finance_research (minutes)
  • research_background (up to 4 hours for frontier effort)

Temporal's guidance is explicit — use a sync Operation "only when its complete execution path is highly reliable, has predictably low latency, and finishes well within the 10-second handler deadline." A sync handler that blows the deadline is killed as a retryable error, and the circuit breaker "trips after 5 consecutive retryable errors, blocking all Operations from the caller to that Endpoint." That failure is shared-fate across the Endpoint, so every Operation here is asynchronous and backed by a thin Workflow.

The tradeoff worth naming: this makes a fast search cost a full Workflow Execution rather than a couple of Actions. We are buying blast-radius safety with per-call cost.

What's here

File Purpose
src/youdotcom_temporal/contract.py The caller-side contract: request types, result types, and YouDotComService. Importing it loads neither the handler nor the Activity layer
src/youdotcom_temporal/nexus.py YouDotComServiceHandler (6 Operations, idempotency-key Workflow-Id derivation) + you_nexus_service_handler()
src/youdotcom_temporal/workflows.py 6 backing Workflows + you_nexus_workflows(), each with a per-Activity start_to_close_timeout and a retry policy matched to cost
src/youdotcom_temporal/plugin.py annotated_types added to the sandbox passthrough (silences 13 late-import warnings)
tests/test_nexus.py Contract, handler, sandbox-registration, Workflow-Id, and timeout/retry tests (no server needed)
tests/test_nexus_integration.py Full round trips against a local dev server: every Operation, a real cross-Namespace call, failure-type propagation, idempotency dedup (mocked You.com)
tests/test_nexus_integration_live.py Live round trips against the real You.com API for the fast Operations (pytest -m integration)
tests/_nexus_caller_workflows.py Caller-side Workflows the round-trip tests drive — the same unwrapped-import shape a real caller has
examples/run_nexus_worker.py Handler-side Worker example + caller-Workflow sketch
README.md, CHANGELOG.md Nexus section and entries

Idempotency

Operation starts are idempotent when the caller opts in. Each request carries the Activity input plus an optional idempotency_key; a key makes the backing Workflow Id deterministic (key and request digest) and starts it with WorkflowIDConflictPolicy.USE_EXISTING, so a retried Nexus StartOperation attaches to the run already in flight instead of paying for a second You.com call. Binding the request into the Id keeps a key inert across different requests, so two callers who pick the same key string cannot receive each other's results. Dedup holds against a running Workflow; a key reused after completion starts a fresh run. Without a key, starts are not deduplicated.

Bug fix worth landing regardless of Nexus adoption

youdotcom_temporal/__init__.py imported the Activity layer eagerly, which imports the You.com SDK and urllib.request. Python imports a parent package before a submodule body runs, so a submodule's own workflow.unsafe.imports_passed_through() block never got the chance to cover it. Verified against SandboxedWorkflowRunner.prepare_workflow — the path Worker.__init__ takes for every registered Workflow:

  • The backing Workflows only registered when YouPlugin was also passed, because the plugin's passthrough list was silently doing the block's job.
  • The documented caller Workflow could not start at all: a caller in another Namespace registers no plugin, so it had no passthrough to inherit.

The public names now resolve through a module __getattr__, so the SDK loads on first attribute access — after any passthrough block has been entered. Public imports are unchanged. This is a latent sharp edge in the Activity layer that the Nexus work exposed; it is not specific to Nexus. (Landed on main separately in v1.0.1.)

Fixes from this PR's review pass

  • The contents Operation used the wrong SDK result model. ContentsOutput wrapped Contents (the search-extraction shape) instead of ContentsResponse (the contents endpoint's response), silently dropping url, title, and metadata from every result element. Live round-trip tests now assert those fields survive.
  • The research Operation accepted background=True it could never honor. With background=True the SDK returns a task handle, which can never validate as the Operation's ResearchResponse result — the caller would have paid for the research task and then received an opaque YouResponseShapeError. research now rejects background=True with a non-retryable YouValidationError before any billable call; research_background is the Operation for that mode.
  • CHANGELOG restructured so the Unreleased section merges cleanly against main's v1.1.0 release instead of nesting under it.

Open question

Do the partner standards expect heartbeating Activities? Cancelling an Operation cancels the backing Workflow, but the Activities do not heartbeat, so an in-flight You.com call runs to completion and is still billed. Fixing it means adding heartbeats to the Activity long-poll loops.

Known limits

  • Cancellation does not reach You.com (see above).
  • nexusrpc is imported but not declared. It arrives transitively via temporalio, which pins nexus-rpc==1.4.0. Declare it under the distribution name nexus-rpc before release — nexusrpc is the import name and not a valid requirement. A range like >=1.4,<2 would conflict when temporalio bumps its pin.

Verification

  • uv run ruff check — clean
  • uv run mypy src — clean (strict)
  • uv run pytest — 124 passed, 12 deselected (integration)
  • uv run pytest tests/test_nexus_integration.py — 13 passed (local dev server, mocked You.com)
  • uv run pytest -m integration tests/test_nexus_integration_live.py — 4 passed (real You.com API)

Out of scope

  • No changes to the Activity implementations, YouPlugin's passthrough list beyond annotated_types, or error mapping.
  • No pyproject.toml dependency changes (deliberately deferred, see limits).

@tyler5673 tyler5673 changed the title feat: Nexus Service support (workflow-backed Operations) spike: Nexus Service support (workflow-backed Operations) Aug 17, 2026
@tyler5673
tyler5673 force-pushed the feat/nexus-service branch 5 times, most recently from 6dd3504 to 503ba91 Compare August 21, 2026 20:54
tyler5673 and others added 2 commits August 21, 2026 14:54
Rebased onto main with PR #15 (SDK 3.1.2 + attribution header) and
PR #16 (extraction parameter) merged. Squashes the nexus spike into
one clean commit.

- YouDotComService exposes all six Activities as async Nexus Operations
- contract.py holds the Nexus contract with SDK response models
- workflows.py ships six thin backing Workflows with per-Activity ceilings
- Idempotency key support for deduplication of Nexus StartOperation retries
- Unit tests covering contract, handler, and sandbox registration
- Integration tests for live Nexus round-trip (gated behind -m integration)

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
…orrect timeout_s comment

- contract.py, nexus.py: SDK 3.1.2 shipped lazy imports (PEP 562), so the
  imports_passed_through() wrapper is belt-and-braces rather than load-bearing.
  Updated comments that referenced DX-776 as future work.
- workflows.py: comment claimed the Activity forwards timeout_s untouched
  but activities.py substitutes 120s when timeout_s is None, preventing the
  SDK effort-based deadline derivation. Corrected to describe actual behavior.
- plugin.py: re-added annotated_types to _PASSTHROUGH_MODULES, eliminating
  13 UserWarning messages about late import under the workflow sandbox.

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
The Activity now forwards timeout_s as-is (PR #17 merged), so the SDK
derives the effort-based deadline itself. Updated the comment that
described the old 120s substitution behavior.

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
@tyler5673
tyler5673 marked this pull request as ready for review September 1, 2026 19:39
ContentsOutput wrapped the wrong SDK model: Contents (the search
extraction shape) instead of ContentsResponse (the contents endpoint
response), silently dropping url, title, and metadata from every
result element.

The research Operation accepted ResearchInput.background=True, but
with background=True the SDK returns a task handle that can never
validate as the Operation's ResearchResponse result. It now rejects
background=True with a non-retryable YouValidationError before any
billable call; research_background is the Operation for that mode.

Also restructures the CHANGELOG Unreleased section so it merges
cleanly against main's 1.1.0 release.

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
@tyler5673 tyler5673 changed the title spike: Nexus Service support (workflow-backed Operations) feat: Nexus Service support (workflow-backed Operations) Sep 1, 2026
@tyler5673
tyler5673 merged commit c4645c6 into main Sep 3, 2026
5 of 6 checks passed
@tyler5673
tyler5673 deleted the feat/nexus-service branch September 3, 2026 04:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant