Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,25 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added
- Nexus Service support: `youdotcom_temporal.nexus.YouDotComService` exposes all six Activities as asynchronous, workflow-backed Nexus Operations callable across Namespace boundaries through a Nexus Endpoint
- `youdotcom_temporal.contract` holds the Nexus contract: a request type per Operation carrying the Activity input plus an optional `idempotency_key`, and result types that are the You.com SDK's own response models, so callers get accurate nested types the SDK maintains. `contents` keeps a thin `ContentsOutput` envelope because the SDK returns a bare list; its elements are `ContentsResponse` models, not the search-extraction `Contents` model, which would have silently dropped `url`, `title`, and `metadata` from every element. Callers must configure `temporalio.contrib.pydantic.pydantic_data_converter`
- Supplying `idempotency_key` makes the backing Workflow Id deterministic and starts it with `WorkflowIDConflictPolicy.USE_EXISTING`, so a retried Nexus StartOperation request attaches to the run already in flight instead of starting a second Workflow and paying for a second You.com call. Deduplication holds against a *running* Workflow; without a key, starts are not deduplicated
- `youdotcom_temporal.workflows` ships six thin backing Workflows (`YouSearchWorkflow`, `YouAnswerWorkflow`, `YouContentsWorkflow`, `YouResearchWorkflow`, `YouFinanceResearchWorkflow`, `YouResearchBackgroundWorkflow`), each wrapping its Activity with a per-Activity `start_to_close_timeout` carrying generous headroom, since a ceiling is a retry backstop rather than a latency target. `search` and `contents` are sized off the 60s per-URL maximum a caller can request via `crawl_timeout`
- The `research` Operation rejects `background=True` with a non-retryable `YouValidationError`. With `background=True` the SDK returns a task handle, which can never validate as the Operation's `ResearchResponse` result—the caller would have paid for the research task and then received an opaque `YouResponseShapeError`. `research_background` is the Operation for background mode
- `you_nexus_service_handler()` and `you_nexus_workflows()` helpers for Worker registration
- `examples/run_nexus_worker.py` handler-side Worker example
- Unit tests covering the Nexus Service contract and handler
- Cross-Namespace round-trip tests against a local dev server (mocked You.com) and live round-trip tests for the fast Operations (`pytest -m integration`)

### Notes
- Nexus is an opt-in layer: importing `youdotcom_temporal.nexus` does not affect Activity-only users
- Every Operation is async/workflow-backed because Nexus sync operations have a 10-second handler deadline that several You.com calls exceed
- The handler Worker needs `YouPlugin` because it registers the Activities the backing Workflows call. Caller Workflows in other Namespaces need neither the plugin nor a sandbox escape
- The research Operations do not retry: each attempt submits a new billable research task, and the previous one keeps running because the You.com API has no way to cancel a submitted request
- Known limits, tracked before release: cancelling an Operation does not stop the in-flight You.com call
- Responses are parsed into SDK models on the handler side. A response that does not match raises a non-retryable `YouResponseShapeError`, because an unguarded parse error inside a Workflow is a Workflow task failure that Temporal would retry indefinitely, hanging the caller

## [1.1.0] — 2026-08-21

### Added
Expand Down
97 changes: 96 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Durable [You.com](https://you.com) search, answer, research, and contents Activities for [Temporal](https://temporal.io).

Exposes You.com API calls as Temporal Activities with proper error mapping, retry semantics, and workflow sandbox support. Ships as a `SimplePlugin` for one-line setup, or as standalone activity functions for manual worker wiring.
Exposes You.com API calls as Temporal Activities with proper error mapping, retry semantics, and workflow sandbox support. Ships as a `SimplePlugin` for one-line setup, or as standalone activity functions for manual worker wiring. An optional Nexus Service (`youdotcom_temporal.nexus`) exposes the same calls as cross-Namespace Operations for teams that want a durable service contract on top of the Activities.

## Installation

Expand Down Expand Up @@ -137,6 +137,100 @@ All activities return JSON-serializable dicts (via `model_dump(mode="json")`).
| `crawl_timeout` | `int` | `10` | Per-URL timeout in seconds (1-60) |
| `max_age` | `int \| None` | `None` | Max cache age in seconds (0 = always re-fetch) |

## Nexus Service

`youdotcom_temporal.nexus.YouDotComService` is a [Temporal Nexus](https://docs.temporal.io/nexus) Service that exposes all six Activities as Operations callable across Namespace boundaries through a Nexus Endpoint. It is an opt-in layer on top of the Activities, so importing it does not affect Activity-only users.

Every Operation is asynchronous and backed by a Workflow (`youdotcom_temporal.workflows`). Nexus synchronous operations must finish within a 10-second handler deadline, which is the wrong shape for these calls: `research` and `finance_research` are multi-step research runs measured in tens of seconds to minutes by design, background research runs up to 4 hours for `frontier`, and `search` and `contents` accept `livecrawl` and `crawl_timeout` (up to 60s per URL), so a caller can legitimately ask for a long call.

A sync handler that misses the deadline fails as a retryable error, and five consecutive retryable errors trip a circuit breaker that blocks *every* Operation on that caller/Endpoint pair for 60 seconds. Routing each Operation through a Workflow removes the cliff and gives the caller durable, observable execution.

Register the handler, the backing Workflows, and the Activities (via `YouPlugin`) on one Worker in the handler Namespace. `YouPlugin` is what registers the Activities the backing Workflows call, so the handler Worker needs it.

Operations return the You.com SDK's own pydantic response models, so both the handler and every caller need Temporal's pydantic data converter.

```python
from temporalio.client import Client
from temporalio.contrib.pydantic import pydantic_data_converter
from temporalio.worker import Worker
from youdotcom_temporal import YouPlugin
from youdotcom_temporal.nexus import you_nexus_service_handler
from youdotcom_temporal.workflows import you_nexus_workflows

async def main():
client = await Client.connect(
"localhost:7233",
namespace="you-handler",
data_converter=pydantic_data_converter,
)
worker = Worker(
client,
task_queue="you-nexus",
workflows=you_nexus_workflows(),
nexus_service_handlers=[you_nexus_service_handler()],
plugins=[YouPlugin()],
)
await worker.run()
```

Create a Nexus Endpoint targeting that Worker, then call an Operation from a caller Workflow in another Namespace. Callers import from `youdotcom_temporal.contract`, which carries the types and none of the handler. No plugin and no sandbox escape are needed, because the contract wraps its own SDK import, so these are safe at module scope:

```python
from datetime import timedelta
from temporalio import workflow

from youdotcom_temporal.contract import SearchRequest, SearchResponse, YouDotComService
from youdotcom_temporal.models import SearchInput

NEXUS_ENDPOINT = "you-nexus-endpoint"

@workflow.defn
class CallerWorkflow:
@workflow.run
async def run(self, query: str) -> SearchResponse:
nexus_client = workflow.create_nexus_client(
service=YouDotComService, endpoint=NEXUS_ENDPOINT
)
out = await nexus_client.execute_operation(
YouDotComService.search,
SearchRequest(
input=SearchInput(query=query, count=10),
# Optional. With a key, a retried StartOperation attaches to the
# Workflow already running instead of starting a second one and
# paying for a second You.com call.
idempotency_key=f"search:{query}",
),
# Must exceed the handler-side worst case: the Activity ceiling
# times the retry policy's maximum_attempts (120s x 3 for search).
schedule_to_close_timeout=timedelta(minutes=10),
)
out.results.web[0].title # typed all the way down
return out
```

| Operation | Request | Result | Backing Workflow |
|---|---|---|---|
| `search` | `SearchRequest` | `SearchResponse` | `YouSearchWorkflow` |
| `answer` | `AnswerRequest` | `AnswerResponse` | `YouAnswerWorkflow` |
| `contents` | `ContentsRequest` | `ContentsOutput` | `YouContentsWorkflow` |
| `research` | `ResearchRequest` | `ResearchResponse` | `YouResearchWorkflow` |
| `finance_research` | `FinanceResearchRequest` | `FinanceResearchResponse` | `YouFinanceResearchWorkflow` |
| `research_background` | `ResearchRequest` | `TaskDetail` | `YouResearchBackgroundWorkflow` |

Every request carries the matching Activity input plus an optional `idempotency_key`. Results are the You.com SDK's own response models, imported from `youdotcom_temporal.contract`, so callers get accurate nested types the SDK maintains. `contents` is the exception: the SDK returns a bare list, which could not gain fields later without breaking callers, so it keeps a thin `ContentsOutput` envelope whose elements are still SDK `ContentsResponse` models. `research` rejects `background=True` with a non-retryable `YouValidationError`, and `research_background` is the Operation for background mode.

Each backing Workflow runs the Activity with a per-Activity `start_to_close_timeout`. A ceiling is a backstop rather than a latency target—it is where the Activity gives up and lets Temporal retry—so each carries generous headroom. `search` is 120s and `contents` 180s, sized off the 60s per-URL maximum a caller can request via `crawl_timeout` (`contents` gets more, since it accepts up to 10 URLs per request). `answer` is 60s, `research` is 10 minutes, `finance_research` is 30 minutes, and `research_background` is 4h15m.

That ceiling is **per attempt**—`search`, `answer`, and `contents` retry up to 3 times, so size the caller's `schedule_to_close_timeout` against the ceiling times the attempt count. The research Operations do not retry: each attempt submits a new billable research task, and the previous one keeps running.

See the [Temporal Python Nexus quickstart](https://docs.temporal.io/develop/python/nexus/quickstart) for Endpoint and caller-Namespace setup, and `examples/run_nexus_worker.py` for a runnable handler-side Worker.

**Known limits (draft):**

- **Cancellation does not reach You.com, and cannot.** The You.com API exposes no cancellation—the research surface is `POST /v1/research` plus two GETs to poll or stream, with no DELETE—so a submitted request runs to completion and is billed regardless. Cancelling an Operation frees the backing Workflow and the Worker slot, nothing upstream.
- **Idempotency is opt-in and bounded.** Supplying `idempotency_key` deduplicates against a Workflow that is still *running*, which covers the StartOperation-retry case. A key reused after the first Operation completed starts a fresh run. Without a key, starts are not deduplicated at all.
- **An unparseable response fails the Operation.** Results are parsed into SDK models on the handler side, and a response that does not match raises a non-retryable `YouResponseShapeError` rather than hanging the caller.

## Error handling

| HTTP status | Error type | Retryable? |
Expand Down Expand Up @@ -171,6 +265,7 @@ See the [`examples/`](examples/) directory:
- `run_worker.py` - starts a worker with `YouPlugin`
- `run_workflow.py` - executes the search workflow
- `run_background_research_workflow.py` - executes the background research workflow
- `run_nexus_worker.py` - handler-side Worker hosting the `YouDotCom` Nexus Service

```bash
# Terminal 1
Expand Down
108 changes: 108 additions & 0 deletions examples/run_nexus_worker.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
"""Handler-side Worker for the YouDotCom Nexus Service.

This Worker hosts the Nexus Service handler, the backing Workflows, and the
You.com Activities (registered by ``YouPlugin``). Run it in the *handler*
Namespace, then point a Nexus Endpoint at this Worker's Task Queue so callers in
other Namespaces can reach the Operations.

``YouPlugin`` registers the Activities the backing Workflows call, so the
handler Worker needs it. Operations return the You.com SDK's pydantic response
models, so the Client needs ``pydantic_data_converter`` -- on the caller side
too.

Setup (two terminals, local dev server with Nexus enabled)::

# 1. Start the Temporal dev server (Nexus is enabled by default).
temporal server start-dev

# 2. Create a handler Namespace and a Nexus Endpoint targeting this Worker.
temporal operator namespace create --namespace you-handler
temporal operator nexus endpoint create \
--name you-nexus-endpoint \
--target-namespace you-handler \
--target-task-queue you-nexus

# 3. Run this handler Worker (terminal 2).
YDC_API_KEY=your-key python examples/run_nexus_worker.py

A caller Workflow in a different Namespace invokes an Operation through the
Endpoint. Callers import from ``youdotcom_temporal.contract``, which carries the
types and none of the handler. No plugin and no sandbox escape are needed, and
the caller's Client needs the same pydantic converter this Worker uses::

from datetime import timedelta
from temporalio import workflow

from youdotcom_temporal.contract import (
SearchRequest,
SearchResponse,
YouDotComService,
)
from youdotcom_temporal.models import SearchInput

NEXUS_ENDPOINT = "you-nexus-endpoint"

@workflow.defn
class CallerWorkflow:
@workflow.run
async def run(self, query: str) -> SearchResponse:
nexus_client = workflow.create_nexus_client(
service=YouDotComService, endpoint=NEXUS_ENDPOINT
)
# Exceeds the handler-side worst case for search
# (_SEARCH_STC 120s x 3 attempts).
out = await nexus_client.execute_operation(
YouDotComService.search,
SearchRequest(
input=SearchInput(query=query, count=10),
# Optional: a retried StartOperation attaches to the run
# already in flight instead of paying for a second call.
idempotency_key=f"search:{query}",
),
schedule_to_close_timeout=timedelta(minutes=10),
)
out.results.web[0].title # typed all the way down
return out

See the Temporal Python Nexus quickstart for the full caller-side setup:
https://docs.temporal.io/develop/python/nexus/quickstart
"""

from __future__ import annotations

import asyncio

from temporalio.client import Client
from temporalio.contrib.pydantic import pydantic_data_converter
from temporalio.worker import Worker

from youdotcom_temporal import YouPlugin
from youdotcom_temporal.nexus import you_nexus_service_handler
from youdotcom_temporal.workflows import you_nexus_workflows

HANDLER_NAMESPACE = "you-handler"
TASK_QUEUE = "you-nexus"


async def main() -> None:
client = await Client.connect(
"localhost:7233",
namespace=HANDLER_NAMESPACE,
data_converter=pydantic_data_converter,
)
worker = Worker(
client,
task_queue=TASK_QUEUE,
workflows=you_nexus_workflows(),
nexus_service_handlers=[you_nexus_service_handler()],
plugins=[YouPlugin()],
)
print(
f"Handler Worker started in namespace {HANDLER_NAMESPACE!r} "
f"on task queue {TASK_QUEUE!r} (YouDotCom Nexus Service)"
)
await worker.run()


if __name__ == "__main__":
asyncio.run(main())
Loading
Loading