diff --git a/docs/0-13-0/assets/favicon.svg b/docs/0-13-0/assets/favicon.svg new file mode 100644 index 000000000..1312d00ad --- /dev/null +++ b/docs/0-13-0/assets/favicon.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/docs/0-13-0/assets/iii-black.svg b/docs/0-13-0/assets/iii-black.svg new file mode 100644 index 000000000..5cdd91eb2 --- /dev/null +++ b/docs/0-13-0/assets/iii-black.svg @@ -0,0 +1,9 @@ + + + + + + + + + \ No newline at end of file diff --git a/docs/0-13-0/assets/iii-white.svg b/docs/0-13-0/assets/iii-white.svg new file mode 100644 index 000000000..1449f07b3 --- /dev/null +++ b/docs/0-13-0/assets/iii-white.svg @@ -0,0 +1,19 @@ + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/0-13-0/changelog/0-11-0/everything-is-a-worker.mdx b/docs/0-13-0/changelog/0-11-0/everything-is-a-worker.mdx new file mode 100644 index 000000000..7d1aebecc --- /dev/null +++ b/docs/0-13-0/changelog/0-11-0/everything-is-a-worker.mdx @@ -0,0 +1,342 @@ +--- +title: 'Everything is a Worker' +description: 'Why we renamed modules to workers and how to migrate.' +--- + +## Why this change + +iii is built on three primitives: **Workers**, **Functions**, and **Triggers**. Every concept in the system maps to one of these: + +- **Workers** are long-running processes that connect to the engine and provide capabilities -- HTTP serving, queue processing, cron scheduling, state management, stream handling, and anything else the engine needs to operate. +- **Functions** are units of work. A worker registers functions, and any other worker (or the engine itself) can invoke them by ID. +- **Triggers** are the rules that cause functions to fire -- an HTTP request, a cron schedule, a queue message, a state change. + +Before this release, the engine-internal components that provide capabilities (HTTP, queues, cron, etc.) were called "modules". Meanwhile, the remote SDK processes that register functions were called "workers". This created a confusing split: two different names for things that serve the same role -- connecting to the engine and doing work. + +Modules **are** workers. They run alongside the engine, register functions, react to triggers, and provide capabilities to the rest of the system. The only difference was the name. + +By renaming modules to workers, every component in the system uses the same vocabulary. The engine config lists `workers:`. The SDK connects `workers`. The docs describe `workers`. There is one concept, one name, and one mental model. + +The previous `Worker` struct (representing a connected SDK runtime over WebSocket) has been renamed to `WorkerConnection` to avoid collision. + +--- + +## What changed + +### Config YAML + +The top-level `modules:` key is now `workers:`, and each entry uses `name:` with a short identifier instead of `class:` with a Rust-style path. + +```yaml +# Before +modules: + - class: modules::stream::StreamModule + config: + port: 3112 + +# After +workers: + - name: iii-stream + config: + port: 3112 +``` + +The `modules:` key is still accepted for backward compatibility, but `workers:` is the canonical form going forward. + +#### Worker name mapping + +| Old `class:` value | New `name:` value | +|---|---| +| `modules::worker::WorkerModule` | `iii-worker-manager` | +| `modules::stream::StreamModule` | `iii-stream` | +| `modules::state::StateModule` | `iii-state` | +| `modules::api::RestApiModule` | `iii-http` | +| `modules::pubsub::PubSubModule` | `iii-pubsub` | +| `modules::observability::OtelModule` | `iii-observability` | +| `modules::queue::QueueModule` | `iii-queue` | +| `modules::http_functions::HttpFunctionsModule` | `iii-http-functions` | +| `modules::cron::CronModule` | `iii-cron` | +| `modules::shell::ExecModule` | `iii-exec` | +| `modules::bridge_client::BridgeClientModule` | `iii-bridge` | + +#### Adapter name mapping + +Adapter entries inside `config.adapter` also switched from `class:` to `name:` with short identifiers: + +| Old adapter `class:` | New adapter `name:` | +|---|---| +| `modules::stream::adapters::KvStore` | `kv` | +| `modules::stream::adapters::RedisAdapter` | `redis` | +| `modules::stream::adapters::Bridge` | `bridge` | +| `modules::state::adapters::KvStore` | `kv` | +| `modules::state::adapters::RedisAdapter` | `redis` | +| `modules::state::adapters::Bridge` | `bridge` | +| `modules::pubsub::LocalAdapter` | `local` | +| `modules::pubsub::RedisAdapter` | `redis` | +| `modules::queue::BuiltinQueueAdapter` | `builtin` | +| `modules::queue::RedisAdapter` | `redis` | +| `modules::queue::RabbitMQAdapter` | `rabbitmq` | +| `modules::queue::adapters::Bridge` | `bridge` | +| `modules::cron::KvCronAdapter` | `kv` | +| `modules::cron::RedisCronAdapter` | `redis` | + +#### Full config example + + + +```yaml +modules: + - class: modules::stream::StreamModule + config: + port: 3112 + adapter: + class: modules::stream::adapters::KvStore + config: + store_method: file_based + file_path: ./data/stream_store + + - class: modules::state::StateModule + config: + adapter: + class: modules::state::adapters::KvStore + config: + store_method: file_based + file_path: ./data/state_store.db + + - class: modules::api::RestApiModule + config: + port: 3111 + + - class: modules::observability::OtelModule + config: + enabled: true + + - class: modules::queue::QueueModule + config: + adapter: + class: modules::queue::BuiltinQueueAdapter + + - class: modules::pubsub::PubSubModule + config: + adapter: + class: modules::pubsub::LocalAdapter + + - class: modules::cron::CronModule + config: + adapter: + class: modules::cron::KvCronAdapter +``` + + +```yaml +workers: + - name: iii-stream + config: + port: 3112 + adapter: + name: kv + config: + store_method: file_based + file_path: ./data/stream_store + + - name: iii-state + config: + adapter: + name: kv + config: + store_method: file_based + file_path: ./data/state_store.db + + - name: iii-http + config: + port: 3111 + + - name: iii-observability + config: + enabled: true + + - name: iii-queue + config: + adapter: + name: builtin + + - name: iii-pubsub + config: + adapter: + name: local + + - name: iii-cron + config: + adapter: + name: kv +``` + + + +--- + +### Rust API + +#### Crate module path + +The `modules` Rust module has been renamed to `workers`, and the trait definition file moved from `module.rs` to `traits.rs`. + +```rust +// Before +use iii::modules::config::EngineBuilder; +use iii::modules::module::Module; +use iii::modules::registry::ModuleRegistration; + +// After +use iii::workers::config::EngineBuilder; // also re-exported as iii::EngineBuilder +use iii::workers::traits::Worker; +use iii::workers::registry::WorkerRegistration; +``` + +#### Trait renames + +| Before | After | +|---|---| +| `trait Module` | `trait Worker` | +| `trait ConfigurableModule` | `trait ConfigurableWorker` | +| `Module::make_module()` | `Worker::make_worker()` | +| `Module::create() -> Box` | `Worker::create() -> Box` | +| `ConfigurableModule::DEFAULT_ADAPTER_CLASS` | `ConfigurableWorker::DEFAULT_ADAPTER_NAME` | +| `ConfigurableModule::default_adapter_class()` | `ConfigurableWorker::default_adapter_name()` | +| `ConfigurableModule::adapter_class_from_config()` | `ConfigurableWorker::adapter_name_from_config()` | + +#### Type alias and struct renames + +| Before | After | +|---|---| +| `ModuleFuture` | `WorkerFuture` | +| `ModuleFactory` | `WorkerFactory` | +| `ModuleInfo` | `WorkerInfo` | +| `ModuleRegistry` | `WorkerRegistry` | +| `ModuleEntry` | `WorkerEntry` | +| `ModuleRegistration` | `WorkerRegistration` | + +#### Macro renames + +| Before | After | +|---|---| +| `register_module!(class, Type, ...)` | `register_worker!(name, Type, ...)` | + +#### EngineBuilder API + +| Before | After | +|---|---| +| `EngineBuilder::add_module(class, config)` | `EngineBuilder::add_worker(name, config)` | +| `EngineBuilder::register_module::(class)` | `EngineBuilder::register_worker::(name)` | + +#### Implementation struct renames + +| Before | After | +|---|---| +| `WorkerModule` | `WorkerManager` | +| `StreamCoreModule` | `StreamWorker` | +| `StateCoreModule` | `StateWorker` | +| `RestApiCoreModule` | `HttpWorker` | +| `PubSubCoreModule` | `PubSubWorker` | +| `OtelModule` | `ObservabilityWorker` | +| `QueueCoreModule` | `QueueWorker` | +| `HttpFunctionsModule` | `HttpFunctionsWorker` | +| `CronCoreModule` | `CronWorker` | +| `ExecCoreModule` | `ExecWorker` | +| `BridgeClientModule` | `BridgeClientWorker` | +| `TelemetryModule` | `TelemetryWorker` | +| `EngineFunctionsModule` | `EngineFunctionsWorker` | +| `ExternalModule` | `ExternalWorker` | + +#### WorkerConnection (was Worker) + +The `Worker` struct that represented a connected remote SDK runtime has been renamed to `WorkerConnection` and moved to a separate module. + +| Before | After | +|---|---| +| `crate::workers::Worker` | `crate::worker_connections::WorkerConnection` | +| `crate::workers::WorkerRegistry` | `crate::worker_connections::WorkerConnectionRegistry` | +| `crate::workers::WorkerStatus` | `crate::worker_connections::WorkerConnectionStatus` | +| `crate::workers::WorkerTelemetryMeta` | `crate::worker_connections::WorkerConnectionTelemetryMeta` | + +--- + +## Migration examples + +### Custom worker + + + +```rust +use iii::modules::module::Module; + +pub struct MyModule { /* ... */ } + +#[async_trait] +impl Module for MyModule { + fn name(&self) -> &'static str { "MyModule" } + async fn create(engine: Arc, config: Option) -> anyhow::Result> { + Ok(Box::new(MyModule { /* ... */ })) + } + async fn initialize(&self) -> anyhow::Result<()> { Ok(()) } +} + +iii::register_module!("my::custom::Module", MyModule, enabled_by_default = true); +``` + + +```rust +use iii::workers::traits::Worker; + +pub struct MyWorker { /* ... */ } + +#[async_trait] +impl Worker for MyWorker { + fn name(&self) -> &'static str { "MyWorker" } + async fn create(engine: Arc, config: Option) -> anyhow::Result> { + Ok(Box::new(MyWorker { /* ... */ })) + } + async fn initialize(&self) -> anyhow::Result<()> { Ok(()) } +} + +iii::register_worker!("iii-my-custom", MyWorker, enabled_by_default = true); +``` + + + +### Custom adapter + + + +```rust +iii::register_adapter!( + + "modules::my_worker::MyAdapter", + make_adapter +); +``` + + +```rust +iii::register_adapter!( + + name: "my-adapter", + make_adapter +); +``` + + + +--- + +## Migration checklist + +- [ ] Rename `modules:` to `workers:` in all config YAML files +- [ ] Replace `class:` with `name:` using the new short identifiers (see tables above) +- [ ] Replace adapter `class:` with `name:` using short identifiers (`kv`, `redis`, `builtin`, etc.) +- [ ] Update Rust imports from `iii::modules::` to `iii::workers::` +- [ ] Update `iii::modules::module::Module` to `iii::workers::traits::Worker` +- [ ] Replace `register_module!` with `register_worker!` +- [ ] Replace `add_module()` / `register_module()` on `EngineBuilder` with `add_worker()` / `register_worker()` +- [ ] Rename any `*Module` / `*CoreModule` struct references to the new `*Worker` / `*Manager` names +- [ ] Update `crate::workers::Worker` references to `crate::worker_connections::WorkerConnection` diff --git a/docs/0-13-0/changelog/0-11-0/migrated-examples.mdx b/docs/0-13-0/changelog/0-11-0/migrated-examples.mdx new file mode 100644 index 000000000..002ac5dea --- /dev/null +++ b/docs/0-13-0/changelog/0-11-0/migrated-examples.mdx @@ -0,0 +1,71 @@ +--- +title: 'Examples migrated from Motia' +description: 'Side-by-side links to the original Motia source and the iii-sdk port for every example we migrated as part of 0.11.0.' +--- + +These examples were ported from the Motia framework to **`iii-sdk@0.11.0`** following the [Motia → Node](/changelog/0-11-0/migrating-from-motia-js) and [Motia → Python](/changelog/0-11-0/migrating-from-motia-py) migration guides. + +Each row links the **original Motia source** alongside the **iii version** so you can read the same workflow in both shapes — every wrapper stripped, every primitive (`registerFunction`, `registerTrigger`, `iii.trigger`) made explicit. + +## Examples + + + + Order approval workflow that pauses for a human and resumes via webhook. Demonstrates HTTP, durable subscribers, and state-driven workflow control. + + **Before:** [`MotiaDev/motia-examples/.../human-in-the-loop`](https://github.com/MotiaDev/motia-examples/tree/main/examples/foundational/workflow-patterns/human-in-the-loop) + **After:** [`iii-hq/examples/human-in-the-loop`](https://github.com/iii-hq/examples/tree/main/human-in-the-loop) + + + + Todo REST API with real-time stream mirror, durable topics for notifications/analytics/achievements, and cron jobs for cleanup and stats. + + **Before:** [`MotiaDev/motia-examples/.../todo-app`](https://github.com/MotiaDev/motia-examples/tree/main/examples/foundational/api-patterns/todo-app) + **After:** [`iii-hq/examples/todo-app`](https://github.com/iii-hq/examples/tree/main/todo-app) + + + + AI chat agent with conversation memory, web search tools, and live streaming responses through `stream::set` updates. + + **Before:** [`MotiaDev/motia-examples/.../ai-chat-agent`](https://github.com/MotiaDev/motia-examples/tree/main/examples/foundational/ai-chat-agent) + **After:** [`iii-hq/examples/ai-chat-agent`](https://github.com/iii-hq/examples/tree/main/ai-chat-agent) + + + + Multi-step property search agent in Python (`iii-sdk` async) — scraping, enrichment, neighbourhood analysis, and market trends as separate workers. + + **Before:** [`MotiaDev/motia-examples/.../property-search-agent`](https://github.com/MotiaDev/motia-examples/tree/main/examples/agentic/property-search-agent) + **After:** [`iii-hq/examples/property-search-agent`](https://github.com/iii-hq/examples/tree/main/property-search-agent) + + + +## What stayed the same + +The business logic, topic names, function IDs, and trigger semantics carry over almost verbatim. If you can read the Motia version you can read the iii version. + +## What changed + +| Concern | Motia | iii-sdk@0.11.0 | +| --- | --- | --- | +| Setup | `motia` framework + auto file scan | `registerWorker(...)` + `import './handlers/...'` side effects in `src/main.ts` | +| HTTP responses | `{ status, body }` | `{ status_code, body }` | +| Path / query | `params`, `query` | `path_params`, `query_params` (values are `string \| string[]`) | +| State | `stateManager.get/set/list` | `iii.trigger({ function_id: 'state::get' \| 'state::set' \| 'state::list' \| 'state::delete', ... })` | +| Streams | `Stream` class | `iii.trigger({ function_id: 'stream::set' \| 'stream::delete' \| 'stream::list' \| ..., ... })` | +| Queue publish | `emit({ topic, data })` | `iii.trigger({ function_id: 'iii::durable::publish', payload: { topic, data } })` | +| Queue subscriber | `EventConfig.subscribes` | `registerTrigger({ type: 'durable:subscriber', config: { topic } })` | +| Cron expression | 5-field `cron` | 7-field `expression` (sec min hour dom month dow year) | +| HTTP path style | `/orders/validate` | `/orders/validate` (leading `/` is **required** in iii) | + +## Try them locally + +Each folder ships with its own `iii-config.yaml`, `package.json`/`pyproject.toml`, and a smoke-test script (`test-*.sh`) so you can run the engine + worker and exercise the full flow in one command. + +```bash +git clone https://github.com/iii-hq/examples +cd examples/todo-app +pnpm install --ignore-workspace +iii --config ./iii-config.yaml +# then in another terminal +bash test-todo-flow.sh +``` diff --git a/docs/0-13-0/changelog/0-11-0/migrating-from-motia-js.mdx b/docs/0-13-0/changelog/0-11-0/migrating-from-motia-js.mdx new file mode 100644 index 000000000..dca196573 --- /dev/null +++ b/docs/0-13-0/changelog/0-11-0/migrating-from-motia-js.mdx @@ -0,0 +1,648 @@ +--- +title: 'Migrating from Motia (Node.js)' +description: 'How to move from the Motia framework to iii-sdk directly in Node.js.' +--- + +Motia was a higher-level framework built on top of `iii-sdk`. It handled file scanning, middleware wiring, trigger registration, and production bundling automatically. These conveniences came at a cost: they hid iii's three core primitives — **Workers**, **Functions**, and **Triggers** — behind opaque abstractions that limited what you could build. + +By moving to `iii-sdk` directly, you unlock the full power of the engine: + +- **Add new workers** that register their own functions and triggers, enabling multi-worker orchestration across services, languages, and runtimes. +- **Connect from the browser** using `iii-browser-sdk` with [Worker RBAC](/0-11-0/how-to/worker-rbac) for secure, real-time frontends — no REST layer needed. See [Use iii in the browser](/0-11-0/how-to/use-iii-in-the-browser). +- **Treat Motia as one worker among many** instead of a standalone monolith. Your existing Motia code becomes just another worker in a larger iii deployment. +- **Understand the primitives directly.** Working with `registerFunction`, `registerTrigger`, and `registerWorker` builds a mental model that transfers across all iii SDKs and documentation. + +Before diving into this migration, we recommend reading the [iii documentation](/0-11-0/) to understand Workers, Functions, and Triggers. The [quickstart](/0-11-0/quickstart) and [Everything is a Worker](/changelog/0-11-0/everything-is-a-worker) pages are good starting points. + +--- + +## Step 1 — Update dependencies + +Remove `motia` and add `iii-sdk`. If you need a production bundler, add `esbuild` as a dev dependency. + +```bash +pnpm remove motia +pnpm add iii-sdk@0.11.0 +pnpm add -D esbuild +``` + +| Before (Motia) | After (iii-sdk) | +|---|---| +| `motia` | `iii-sdk@0.11.0` | +| `motia build` | `esbuild` via `bun run esbuild.config.ts` | +| `motia dev && bun run dist/index.js` | `bun run --watch src/main.ts` | + +--- + +## Step 2 — Initialize the SDK + +Create `src/lib/iii.ts`. This replaces the implicit connection that Motia managed for you. + +```typescript title="@/lib/iii" +import { Logger, registerWorker } from 'iii-sdk' + +export const iii = registerWorker(process.env.III_URL ?? 'ws://localhost:49134', { + workerName: 'api-worker', +}) + +export const logger = new Logger() +``` + +Every `import { logger } from 'motia'` in your codebase changes to `import { logger } from '@/lib/iii'`. + +--- + +## Step 3 — Migrate handlers + +Motia auto-registered functions and triggers from exported `config` objects. With iii-sdk you call `registerFunction` and `registerTrigger` directly. + +### HTTP + + + +```typescript +import { type Handlers, http, type StepConfig } from 'motia' +import { z } from 'zod' + +export const config = { + name: 'Health', + triggers: [http('GET', '/health', { responseSchema: { 200: z.object({ ok: z.boolean() }) } })], + enqueues: [], +} as const satisfies StepConfig + +export const handler: Handlers = async (_input, _ctx) => { + return { status: 200, body: { ok: true } } +} +``` + + +```typescript +import { iii } from '@/lib/iii' + +const ref = iii.registerFunction('health', async () => { + return { status_code: 200, body: { ok: true } } +}) + +iii.registerTrigger({ + type: 'http', + function_id: ref.id, + config: { api_path: '/health', http_method: 'GET' }, +}) +``` + + + +### HTTP with middleware + + + +```typescript +import { type Handlers, http, type StepConfig } from 'motia' + +export const config = { + name: 'List Tags', + triggers: [http('GET', '/tag', { middleware: [requireAuth] })], + enqueues: [], +} as const satisfies StepConfig + +export const handler: Handlers = async (input, ctx) => { + const { sub: userId, orgId } = input.tokenInfo + return { status: 200, body: { tags: allTags } } +} +``` + + +```typescript +import { iii } from '@/lib/iii' + +const ref = iii.registerFunction('list-tags', async (input) => { + const { sub: userId, orgId } = input.tokenInfo + return { status_code: 200, body: { tags: allTags } } +}) + +iii.registerTrigger({ + type: 'http', + function_id: ref.id, + config: { + api_path: '/tag', + http_method: 'GET', + middleware_function_ids: ['middleware::auth'], + }, +}) +``` + +Middleware functions must be registered separately with `registerFunction`. See [Use HTTP middleware](/0-11-0/how-to/use-http-middleware) for details. + + + +### Cron + + + +```typescript +import { cron, type Handlers, type StepConfig } from 'motia' + +export const config = { + name: 'Daily Report', + triggers: [cron('0 0 9 * * * *')], + enqueues: [], +} as const satisfies StepConfig + +export const handler: Handlers = async (_input, _ctx) => { + await generateReport() +} +``` + + +```typescript +import { iii } from '@/lib/iii' + +const ref = iii.registerFunction('daily-report', async () => { + await generateReport() +}) + +iii.registerTrigger({ + type: 'cron', + function_id: ref.id, + config: { expression: '0 0 9 * * * *' }, +}) +``` + + + +### Queue (durable subscriber) + + + +```typescript +import { type Handlers, queue, type StepConfig } from 'motia' + +export const config = { + name: 'Process Order', + triggers: [queue('order.created')], + enqueues: [], +} as const satisfies StepConfig + +export const handler: Handlers = async (input, _ctx) => { + await processOrder(input) +} +``` + + +```typescript +import { iii } from '@/lib/iii' + +const ref = iii.registerFunction('process-order', async (input) => { + await processOrder(input) +}) + +iii.registerTrigger({ + type: 'durable:subscriber', + function_id: ref.id, + config: { topic: 'order.created' }, +}) +``` + + + +To publish to a topic, use `iii.trigger` instead of Motia's `enqueue`: + +```typescript +import { iii } from '@/lib/iii' + +iii.trigger({ + function_id: 'iii::durable::publish', + payload: { topic: 'order.created', data: orderPayload }, +}) +``` + +### State + + + +```typescript +import { type Handlers, state, type StepConfig } from 'motia' + +export const config = { + name: 'On State Change', + triggers: [state()], + enqueues: [], +} as const satisfies StepConfig + +export const handler: Handlers = async (input, _ctx) => { + await handleStateChange(input) +} +``` + + +```typescript +import { iii } from '@/lib/iii' + +const ref = iii.registerFunction('on-state-change', async (input) => { + await handleStateChange(input) +}) + +iii.registerTrigger({ + type: 'state', + function_id: ref.id, + config: {}, +}) +``` + + + +### Stream + + + +```typescript +import { type Handlers, stream, type StepConfig } from 'motia' + +export const config = { + name: 'Chat Message', + triggers: [stream('chat', { groupId: 'room-1' })], + enqueues: [], +} as const satisfies StepConfig + +export const handler: Handlers = async (input, _ctx) => { + await handleChatMessage(input) +} +``` + + +```typescript +import { iii } from '@/lib/iii' + +const ref = iii.registerFunction('chat-message', async (input) => { + await handleChatMessage(input) +}) + +iii.registerTrigger({ + type: 'stream', + function_id: ref.id, + config: { stream_name: 'chat', group_id: 'room-1' }, +}) +``` + + + +### Multiple triggers on one function + +A function can have multiple triggers. This was possible in Motia via the `triggers` array, and maps directly to multiple `registerTrigger` calls sharing the same `function_id`: + +```typescript +const ref = iii.registerFunction('order-handler', async (input) => { + await handleOrder(input) +}) + +iii.registerTrigger({ + type: 'http', + function_id: ref.id, + config: { api_path: '/orders', http_method: 'POST' }, +}) + +iii.registerTrigger({ + type: 'durable:subscriber', + function_id: ref.id, + config: { topic: 'order.retry' }, +}) +``` + +### Stream and State operations + +In Motia, you used the `Stream` class and `stateManager` to read and write data. Under the hood, these called `iii.trigger()` with built-in function IDs (`stream::get`, `state::set`, etc.). With iii-sdk, you call `iii.trigger()` directly. See the [Stream](/0-11-0/workers/iii-stream) and [State](/0-11-0/workers/iii-state) worker docs for the full list of operations. + + + +```typescript +import { iii } from '@/lib/iii' + +const todo = await iii.trigger({ + function_id: 'stream::get', + payload: { stream_name: 'todos', group_id: 'user-1', item_id: 'todo-1' }, +}) + +await iii.trigger({ + function_id: 'stream::set', + payload: { stream_name: 'todos', group_id: 'user-1', item_id: 'todo-1', data: { title: 'Buy milk' } }, +}) + +await iii.trigger({ + function_id: 'stream::delete', + payload: { stream_name: 'todos', group_id: 'user-1', item_id: 'todo-1' }, +}) + +const items = await iii.trigger({ + function_id: 'stream::list', + payload: { stream_name: 'todos', group_id: 'user-1' }, +}) +``` + + +```typescript +import { iii } from '@/lib/iii' + +const theme = await iii.trigger({ + function_id: 'state::get', + payload: { scope: 'settings', key: 'theme' }, +}) + +await iii.trigger({ + function_id: 'state::set', + payload: { scope: 'settings', key: 'theme', value: 'dark' }, +}) + +await iii.trigger({ + function_id: 'state::delete', + payload: { scope: 'settings', key: 'theme' }, +}) + +const allSettings = await iii.trigger({ + function_id: 'state::list', + payload: { scope: 'settings' }, +}) +``` + + + +Additional function IDs: `stream::update`, `stream::list_groups`, `stream::send` for streams; `state::update`, `state::list_groups` for state. + +### Stream lifecycle hooks (`onJoin` / `onLeave`) + +In Motia, `*.stream.ts` files could define `onJoin` and `onLeave` callbacks inside the `StreamConfig`. The framework registered a single shared function for each hook and dispatched by stream name internally. + +With iii-sdk, you register these as regular functions with `stream:join` and `stream:leave` triggers. The handler receives a `StreamJoinLeaveEvent` with `{ stream_name, group_id, id, context }`. + + + +```typescript +import { type StreamConfig } from 'motia' + +export const config: StreamConfig = { + name: 'todo', + schema: todoSchema, + baseConfig: { storageType: 'default' }, + + onJoin: async (subscription, _context, authContext) => { + return { unauthorized: false } + }, + + onLeave: async (subscription, _context, authContext) => { + // cleanup logic + }, +} +``` + + +```typescript +import { iii } from '@/lib/iii' + +iii.registerFunction('stream::on-join', async (event) => { + const { stream_name, group_id, id, context } = event + return { unauthorized: false } +}) + +iii.registerTrigger({ + type: 'stream:join', + function_id: 'stream::on-join', + config: {}, +}) + +iii.registerFunction('stream::on-leave', async (event) => { + const { stream_name, group_id, id, context } = event + // cleanup logic +}) + +iii.registerTrigger({ + type: 'stream:leave', + function_id: 'stream::on-leave', + config: {}, +}) +``` + + + +If you had multiple streams with different `onJoin` / `onLeave` logic, dispatch by `event.stream_name` inside the handler or register separate function IDs per stream. + +### Stream authentication (`authenticateStream`) + +In Motia, `motia.config.ts` exported an `authenticateStream` function that ran during WebSocket upgrade. With iii-sdk, you register a function directly and reference its ID in the engine's stream module config. + + + +```typescript +// motia.config.ts +import type { AuthenticateStream } from 'motia' + +export const authenticateStream: AuthenticateStream = async (req, context) => { + return { context: { userId: 'sergio' } } +} +``` + + +```typescript +import { iii } from '@/lib/iii' + +iii.registerFunction('stream::authenticate', async (input) => { + const { headers, path, query_params, addr } = input + return { context: { userId: 'sergio' } } +}) +``` + + + +The engine calls this function during WebSocket upgrade. Reference the function ID in your `config.yaml`: + +```yaml +workers: + - name: iii-stream + config: + auth_function: stream::authenticate +``` + +Key differences: + +- Motia used `queryParams` (camelCase); iii-sdk uses `query_params` (snake_case). +- No trigger registration is needed for auth — it is config-driven via `auth_function`. +- The `motia.config.ts` file is no longer needed; delete it. + +--- + +## Step 4 — Update request and response shapes + +### Request properties + +Motia wrapped iii-sdk's HTTP request into friendlier property names. With iii-sdk you receive the raw shape. + +| Property | Motia (`input.*`) | iii-sdk (`input.*`) | +|---|---|---| +| Route params | `params` | `path_params` | +| Query string | `query` | `query_params` | +| JSON body | `body` | `body` | +| Headers | `headers` | `headers` | +| HTTP method | `method` | `method` | +| Request stream | `requestBody` | `request_body` | +| Response stream | `response` | `response` | + +```typescript +// Before (Motia) +const { folderId } = input.params || {} +const parentFolderId = input.query?.parentFolderId + +// After (iii-sdk) +const { folderId } = input.path_params || {} +const parentFolderId = input.query_params?.parentFolderId +``` + +### `query_params` value types + +In iii-sdk, `query_params` values are `string | string[]`. Use a helper to safely extract the first value: + +```typescript +function firstQueryParam( + queryParams: Record | undefined, + name: string, +): string | undefined { + const v = queryParams?.[name] + if (v === undefined) return undefined + return Array.isArray(v) ? v[0] : v +} +``` + +### Response shape + +iii-sdk uses `status_code` instead of `status`. This applies to every handler return and middleware response. + +```typescript +// Before (Motia) +return { status: 200, body: { tags: allTags } } +return { status: 404, body: { error: 'Not found' } } + +// After (iii-sdk) +return { status_code: 200, body: { tags: allTags } } +return { status_code: 404, body: { error: 'Not found' } } +``` + +--- + +## Step 5 — Set up the entry point + +Motia discovered `.step.ts` files automatically. With iii-sdk, you create a single entry point that imports all handler files as side effects. + +Create `src/main.ts`: + +```typescript +import './lib/iii' +import './handlers/health' +import './handlers/auth' +import './handlers/orders/create-order' +import './handlers/orders/list-orders' +import './handlers/reports/daily-report' +// ... import every handler file +``` + +Each import executes the `registerFunction` and `registerTrigger` calls at the module level, registering the function and its triggers with the engine. + +### File renaming + +Rename all `*.step.ts` files to `*.ts`: + +``` +src/steps/health.step.ts → src/handlers/health.ts +src/steps/auth.step.ts → src/handlers/auth.ts +``` + +The directory structure is yours to decide — `handlers/`, `routes/`, `rest/`, or any layout that fits your project. + +--- + +## Step 6 — Development workflow + +Replace Motia's dev command with a direct `bun` watcher. In your `config.yaml`, configure the worker under `iii-exec`: + +```yaml +workers: + - name: iii-exec + config: + exec: + - bun run --watch src/main.ts +``` + +Run the engine, and it will start your worker process with file watching built in. + +--- + +## Step 7 — Production build + +Motia had `motia build`. Replace it with a plain esbuild config. + +Add to `package.json`: + +```json +{ + "scripts": { + "build": "bun run esbuild.config.ts" + } +} +``` + +Create `esbuild.config.ts`: + +```typescript +import * as esbuild from 'esbuild' + +esbuild.build({ + entryPoints: ['src/main.ts'], + outfile: 'dist/index-production.js', + bundle: true, + platform: 'node', + target: ['node22'], + format: 'esm', + minify: true, + sourcemap: true, + external: ['ws'], +}).catch((err) => { + console.error(err) + process.exit(1) +}) +``` + +Key decisions: + +- **`bundle: true`** — all application code and npm dependencies are bundled into a single file. +- **`external: ['ws']`** — the `ws` package has native addons and must remain in `node_modules` at runtime. +- **`sourcemap: true`** — run production with `bun run --enable-source-maps dist/index-production.js` so stack traces map back to TypeScript source. +- **`platform: 'node'`** — Node.js built-ins (`fs`, `path`, `crypto`, etc.) are treated as external. +- **`format: 'esm'`** — matches `"type": "module"` in your `package.json`. + +Update your production config to run the bundled output: + +```yaml +workers: + - name: iii-exec + config: + exec: + - bun run --enable-source-maps dist/index-production.js +``` + +--- + +## Migration checklist + +- [ ] Remove `motia` from dependencies, add `iii-sdk@0.11.0` +- [ ] Add `esbuild` as a dev dependency +- [ ] Create `src/lib/iii.ts` with `registerWorker` and `Logger` +- [ ] Rename all `*.step.ts` files to `*.ts` +- [ ] Replace all `import { ... } from 'motia'` with iii-sdk imports +- [ ] Convert every `config` + `handler` export to `registerFunction` + `registerTrigger` calls +- [ ] Replace `status` with `status_code` in every handler return +- [ ] Replace `params` with `path_params` and `query` with `query_params` +- [ ] Replace `requestBody` with `request_body` in streaming handlers +- [ ] Replace `Stream` and `stateManager` usage with `iii.trigger()` calls or custom wrappers +- [ ] Migrate stream `onJoin` / `onLeave` hooks to `registerFunction` + `registerTrigger` (`stream:join` / `stream:leave`) +- [ ] Migrate `authenticateStream` from `motia.config.ts` to a registered function and set `auth_function` in `config.yaml` +- [ ] Delete `motia.config.ts` +- [ ] Create `src/main.ts` with side-effect imports for every handler +- [ ] Replace `motia dev` with `bun run --watch src/main.ts` in your dev config +- [ ] Replace `motia build` with an esbuild config +- [ ] Test all endpoints, cron jobs, queue subscribers, and stream handlers diff --git a/docs/0-13-0/changelog/0-11-0/migrating-from-motia-py.mdx b/docs/0-13-0/changelog/0-11-0/migrating-from-motia-py.mdx new file mode 100644 index 000000000..76cd6e7e4 --- /dev/null +++ b/docs/0-13-0/changelog/0-11-0/migrating-from-motia-py.mdx @@ -0,0 +1,667 @@ +--- +title: 'Migrating from Motia (Python)' +description: 'How to move from the Motia framework to iii-sdk directly in Python.' +--- + +Motia was a higher-level framework built on top of `iii-sdk`. It handled file scanning, middleware wiring, trigger registration, and production packaging automatically. These conveniences came at a cost: they hid iii's three core primitives — **Workers**, **Functions**, and **Triggers** — behind opaque abstractions that limited what you could build. + +By moving to `iii-sdk` directly, you unlock the full power of the engine: + +- **Add new workers** that register their own functions and triggers, enabling multi-worker orchestration across services, languages, and runtimes. +- **Connect from the browser** using `iii-browser-sdk` with [Worker RBAC](/0-11-0/how-to/worker-rbac) for secure, real-time frontends — no REST layer needed. See [Use iii in the browser](/0-11-0/how-to/use-iii-in-the-browser). +- **Treat Motia as one worker among many** instead of a standalone monolith. Your existing Motia code becomes just another worker in a larger iii deployment. +- **Understand the primitives directly.** Working with `register_function`, `register_trigger`, and `register_worker` builds a mental model that transfers across all iii SDKs and documentation. + +Before diving into this migration, we recommend reading the [iii documentation](/0-11-0/) to understand Workers, Functions, and Triggers. The [quickstart](/0-11-0/quickstart) and [Everything is a Worker](/changelog/0-11-0/everything-is-a-worker) pages are good starting points. + +For the Node / TypeScript migration, see [Migrating from Motia (Node.js)](/changelog/0-11-0/migrating-from-motia-js). + +--- + +## Step 1 — Update dependencies + +Remove `motia` and add `iii-sdk`. + +```bash +pip uninstall motia +pip install iii-sdk==0.11.0 +``` + +| Before (Motia) | After (iii-sdk) | +|---|---| +| `motia` (pip) | `iii-sdk==0.11.0` (pip) | +| `motia build` | `python -m build` or a container image | +| `motia dev` | `python -m src.main` with a file watcher | + +The Python package is distributed as `iii-sdk` on PyPI but imported as `iii` (e.g., `from iii import register_worker`). + +--- + +## Step 2 — Initialize the SDK + +Create `src/lib/iii_client.py`. This replaces the implicit connection that Motia managed for you. + +```python +import os +from iii import Logger, register_worker, InitOptions + +iii = register_worker( + address=os.environ.get("III_URL", "ws://localhost:49134"), + options=InitOptions(worker_name="api-worker"), +) + +logger = Logger() +``` + +Every `from motia import logger` in your codebase changes to `from src.lib.iii_client import logger`. + +--- + +## Step 3 — Register functions and triggers directly + +Motia auto-registered functions and triggers from exported `config` objects. With `iii-sdk` Python, you call `iii.register_function` and `iii.register_trigger` directly at module scope — there's no helper to build. Each handler module registers its function and trigger(s) when imported. This mirrors the Motia `config` + `handler` structure without the implicit discovery. + +Continue to Step 4 to see the handler patterns. + +`iii-sdk` Python supports engine-native HTTP middleware via `middleware_function_ids`. See [Use HTTP middleware](/0-11-0/how-to/use-http-middleware) for the full pattern. + +--- + +## Step 4 — Migrate handlers + +### HTTP + + + +```python +from motia import http, ApiRequest, ApiResponse + +config = { + "name": "Health", + "description": "Health check", + "triggers": [http("GET", "/health")], + "enqueues": [], +} + +def handler(_request: ApiRequest) -> ApiResponse: + return ApiResponse(status=200, body={"ok": True}) +``` + + +```python +def health(_data): + return {"statusCode": 200, "body": {"ok": True}} + +iii.register_function("health", health) +iii.register_trigger({ + "type": "http", + "function_id": "health", + "config": {"api_path": "/health", "http_method": "GET"}, +}) +``` + + + +### HTTP with middleware + + + +```python +from motia import http, ApiRequest, ApiResponse + +def require_auth(request: ApiRequest) -> bool: + return request.headers.get("authorization") is not None + +config = { + "name": "ListTags", + "description": "List tags", + "triggers": [http("GET", "/tag", middleware=[require_auth])], + "enqueues": [], +} + +def handler(request: ApiRequest) -> ApiResponse: + org_id = request.headers.get("x-org-id") + return ApiResponse(status=200, body={"tags": list_tags_for_org(org_id)}) +``` + + +```python +from src.middleware.auth import require_auth # registers "middleware::require-auth" + +def list_tags(data): + headers = data.get("headers") or {} + org_id = headers.get("x-org-id") + return {"statusCode": 200, "body": {"tags": list_tags_for_org(org_id)}} + +iii.register_function("list-tags", list_tags) +iii.register_trigger({ + "type": "http", + "function_id": "list-tags", + "config": { + "api_path": "/tag", + "http_method": "GET", + "middleware_function_ids": ["middleware::require-auth"], + }, +}) +``` + + + +`iii-sdk` Python uses engine-native middleware registered as regular functions with a `middleware::` prefix. See [Use HTTP middleware](/0-11-0/how-to/use-http-middleware) for the full pattern. + +### Cron + + + +```python +from motia import cron, logger + +config = { + "name": "DailyReport", + "description": "Generate daily report", + "triggers": [cron("0 0 9 * * * *")], + "enqueues": [], +} + +def handler(input: None) -> None: + generate_report() +``` + + +```python +def daily_report(_data) -> None: + generate_report() + +iii.register_function("daily-report", daily_report) +iii.register_trigger({ + "type": "cron", + "function_id": "daily-report", + "config": {"expression": "0 0 9 * * * *"}, +}) +``` + + + +### Queue (durable subscriber) + + + +```python +from motia import queue, logger +from typing import Any + +config = { + "name": "ProcessOrder", + "description": "Process created orders", + "triggers": [queue("order.created")], + "enqueues": [], +} + +def handler(input: Any) -> None: + process_order(input) +``` + + +```python +def process_order(data) -> None: + process_order_impl(data) + +iii.register_function("process-order", process_order) +iii.register_trigger({ + "type": "durable:subscriber", + "function_id": "process-order", + "config": {"topic": "order.created"}, +}) +``` + + + +To publish to a topic, use `iii.trigger` instead of Motia's `enqueue`: + +```python +iii.trigger({ + "function_id": "iii::durable::publish", + "payload": {"topic": "order.created", "data": order_payload}, +}) +``` + +### State + + + +```python +from motia import StateTriggerInput, state, logger + +config = { + "name": "OnStateChange", + "description": "React to state changes", + "triggers": [state()], + "enqueues": [], +} + +def handler(input: StateTriggerInput) -> None: + handle_state_change(input) +``` + + +```python +def on_state_change(data) -> None: + handle_state_change(data) + +iii.register_function("on-state-change", on_state_change) +iii.register_trigger({ + "type": "state", + "function_id": "on-state-change", + "config": {}, +}) +``` + + + +### Stream + + + +```python +from motia import StreamTriggerInput, stream, logger + +config = { + "name": "ChatMessage", + "description": "Handle chat messages", + "triggers": [stream("chat", group_id="room-1")], + "enqueues": [], +} + +def handler(input: StreamTriggerInput) -> None: + handle_chat_message(input) +``` + + +```python +def chat_message(data) -> None: + handle_chat_message(data) + +iii.register_function("chat-message", chat_message) +iii.register_trigger({ + "type": "stream", + "function_id": "chat-message", + "config": {"stream_name": "chat", "group_id": "room-1"}, +}) +``` + + + +### Multiple triggers on one function + +A function can be driven by multiple triggers. This was possible in Motia via the `triggers` array, and maps directly to multiple `register_trigger` calls for the same `function_id`: + +```python +def order_handler(data) -> None: + handle_order(data) + +iii.register_function("order-handler", order_handler) +iii.register_trigger({ + "type": "http", + "function_id": "order-handler", + "config": {"api_path": "/orders", "http_method": "POST"}, +}) +iii.register_trigger({ + "type": "durable:subscriber", + "function_id": "order-handler", + "config": {"topic": "order.retry"}, +}) +``` + +### Stream and State operations + +In Motia, you used the `Stream` class and `stateManager` singleton to read and write data. Under the hood, these called `iii.trigger()` with built-in function IDs (`stream::get`, `state::set`, etc.). With iii-sdk, you call `iii.trigger()` directly. See the [Stream](/0-11-0/workers/iii-stream) and [State](/0-11-0/workers/iii-state) worker docs for the full list of operations. + + + +```python +from src.lib.iii_client import iii + +todo = iii.trigger({ + "function_id": "stream::get", + "payload": {"stream_name": "todos", "group_id": "user-1", "item_id": "todo-1"}, +}) + +iii.trigger({ + "function_id": "stream::set", + "payload": {"stream_name": "todos", "group_id": "user-1", "item_id": "todo-1", "data": {"title": "Buy milk"}}, +}) + +iii.trigger({ + "function_id": "stream::delete", + "payload": {"stream_name": "todos", "group_id": "user-1", "item_id": "todo-1"}, +}) + +items = iii.trigger({ + "function_id": "stream::list", + "payload": {"stream_name": "todos", "group_id": "user-1"}, +}) +``` + + +```python +from src.lib.iii_client import iii + +theme = iii.trigger({ + "function_id": "state::get", + "payload": {"scope": "settings", "key": "theme"}, +}) + +iii.trigger({ + "function_id": "state::set", + "payload": {"scope": "settings", "key": "theme", "value": "dark"}, +}) + +iii.trigger({ + "function_id": "state::delete", + "payload": {"scope": "settings", "key": "theme"}, +}) + +all_settings = iii.trigger({ + "function_id": "state::list", + "payload": {"scope": "settings"}, +}) +``` + + + +Additional function IDs: `stream::update`, `stream::list_groups`, `stream::send` for streams; `state::update`, `state::list_groups` for state. + +### Stream lifecycle hooks (`onJoin` / `onLeave`) + +In Motia, stream files could define `onJoin` and `onLeave` callbacks inside the stream config. The framework registered a single shared function for each hook and dispatched by stream name internally. + +With iii-sdk, you register these as regular functions with `stream:join` and `stream:leave` triggers. The handler receives a dict with `stream_name`, `group_id`, `id`, and `context`. + + + +```python +config = { + "name": "todo", + "schema": todo_schema, + "base_config": {"storage_type": "default"}, +} + +def on_join(subscription, context, auth_context): + return {"unauthorized": False} + +def on_leave(subscription, context, auth_context): + pass # cleanup logic +``` + + +```python +from src.lib.iii_client import iii + +def stream_on_join(event): + stream_name = event.get("stream_name") + group_id = event.get("group_id") + item_id = event.get("id") + context = event.get("context") + return {"unauthorized": False} + +iii.register_function("stream::on-join", stream_on_join) +iii.register_trigger({ + "type": "stream:join", + "function_id": "stream::on-join", + "config": {}, +}) + +def stream_on_leave(event): + stream_name = event.get("stream_name") + group_id = event.get("group_id") + item_id = event.get("id") + context = event.get("context") + # cleanup logic + +iii.register_function("stream::on-leave", stream_on_leave) +iii.register_trigger({ + "type": "stream:leave", + "function_id": "stream::on-leave", + "config": {}, +}) +``` + + + +If you had multiple streams with different `onJoin` / `onLeave` logic, dispatch by `event["stream_name"]` inside the handler or register separate function IDs per stream. + +### Stream authentication (`authenticateStream`) + +In Motia, `motia_config.py` (or `motia.config.ts` for JS projects) exported an authentication function that ran during WebSocket upgrade. With iii-sdk, you register a function directly and reference its ID in the engine's stream module config. + + + +```python +# motia_config.py +def authenticate_stream(req, context): + return {"context": {"userId": "sergio"}} +``` + + +```python +from src.lib.iii_client import iii + +def stream_authenticate(data): + headers = data.get("headers") + path = data.get("path") + query_params = data.get("query_params") + addr = data.get("addr") + return {"context": {"userId": "sergio"}} + +iii.register_function("stream::authenticate", stream_authenticate) +``` + + + +The engine calls this function during WebSocket upgrade. Reference the function ID in your `config.yaml`: + +```yaml +workers: + - name: iii-stream + config: + auth_function: stream::authenticate +``` + +Key differences: + +- Motia used `queryParams` (camelCase); iii-sdk uses `query_params` (snake_case). +- No trigger registration is needed for auth — it is config-driven via `auth_function`. +- The `motia_config.py` file is no longer needed; delete it. + +--- + +## Step 5 — Update request and response shapes + +### Request properties + +Motia wrapped `iii-sdk`'s HTTP request into friendlier property names. With `iii-sdk` you receive the raw shape. + +| Property | Motia (`input.*`) | iii-sdk (`input.*`) | +|---|---|---| +| Route params | `params` | `path_params` | +| Query string | `query` | `query_params` | +| JSON body | `body` | `body` | +| Headers | `headers` | `headers` | +| HTTP method | `method` | `method` | +| Request stream | `requestBody` | `request_body` | +| Response stream | `response` | `response` | + +Motia Python's `ApiRequest` already exposed `path_params`, `query_params`, `body`, `headers`, `method`. When switching to `iii-sdk`, handlers receive those fields in a **plain dict** instead of a Pydantic model, so attribute access (`request.path_params`) becomes dict-key access (`data["path_params"]`). + +```python +# Before (Motia — Pydantic model, attribute access) +folder_id = request.path_params.get("folderId") +parent_folder_id = request.query_params.get("parentFolderId") + +# After (iii-sdk — raw dict, key access) +folder_id = (data.get("path_params") or {}).get("folderId") +parent_folder_id = (data.get("query_params") or {}).get("parentFolderId") +``` + +### `query_params` value types + +In `iii-sdk`, `query_params` values are `str | list[str]`. Use a helper to safely extract the first value: + +```python +def first_query_param(data: dict, name: str) -> str | None: + value = (data.get("query_params") or {}).get(name) + if value is None: + return None + return value[0] if isinstance(value, list) else value +``` + +### Response shape + +`iii-sdk` uses `statusCode` instead of `status`. This applies to every handler return and middleware response. + +```python +# Before (Motia) +return ApiResponse(status=200, body={"tags": all_tags}) +return ApiResponse(status=404, body={"error": "Not found"}) + +# After (iii-sdk) +return {"statusCode": 200, "body": {"tags": all_tags}} +return {"statusCode": 404, "body": {"error": "Not found"}} +``` + +--- + +## Step 6 — Set up the entry point + +Motia discovered `*_step.py` files automatically. With `iii-sdk` Python, create `src/main.py` that imports every handler module as a side effect. + +```python +# src/main.py +import signal +import time + +from src.lib.iii_client import iii # noqa: F401 — connects on import + +# Import every handler module; each registers its function + trigger at import time. +from src.handlers import health # noqa: F401 +from src.handlers import auth # noqa: F401 +from src.handlers.orders import create_order # noqa: F401 +from src.handlers.orders import list_orders # noqa: F401 +from src.handlers.reports import daily_report # noqa: F401 +# ... import every handler module + + +def _shutdown(*_args) -> None: + iii.shutdown() + raise SystemExit(0) + + +if __name__ == "__main__": + signal.signal(signal.SIGINT, _shutdown) + signal.signal(signal.SIGTERM, _shutdown) + while True: + time.sleep(1) +``` + +Each import executes the `register_function` and `register_trigger` calls at the module level, registering the function and its triggers with the engine. + +### File renaming + +Rename all `*_step.py` files to remove the `_step` suffix: + +``` +src/steps/health_step.py → src/handlers/health.py +src/steps/auth_step.py → src/handlers/auth.py +``` + +The directory structure is yours to decide — `handlers/`, `routes/`, `rest/`, or any layout that fits your project. + +--- + +## Step 7 — Development workflow + +The `iii-exec` worker has a built-in file watcher. In your `config.yaml`, declare `watch` globs alongside the `exec` command — no extra dependencies, no wrapper process: + +```yaml +workers: + - name: iii-exec + config: + watch: + - src/**/*.py + exec: + - python + - -m + - src.main +``` + +Any change matching a `watch` glob restarts the `exec` pipeline. + +--- + +## Step 8 — Production build + +Python has no direct `esbuild` equivalent. The recommended production path is a virtualenv (or container image) with pinned dependencies. + +Add to `pyproject.toml`: + +```toml +[project] +name = "my-worker" +version = "0.1.0" +dependencies = [ + "iii-sdk==0.11.0", +] +``` + +Build and run: + +```bash +pip install . +python -m src.main +``` + +For containerized deploys, a minimal `Dockerfile`: + +```dockerfile +FROM python:3.11-slim +WORKDIR /app +COPY pyproject.toml ./ +RUN pip install --no-cache-dir . +COPY src ./src +CMD ["python", "-m", "src.main"] +``` + +Update your production config to run the installed module (omit `watch` in production): + +```yaml +workers: + - name: iii-exec + config: + exec: + - python + - -m + - src.main +``` + +Unlike `esbuild`, you do not bundle Python sources — the runtime imports modules directly. Ship your source tree (or a wheel) rather than a single file. + +--- + +## Migration checklist + +- [ ] Remove `motia` from `requirements.txt` / `pyproject.toml`, add `iii-sdk==0.11.0` +- [ ] Create `src/lib/iii_client.py` with `register_worker` and `Logger` +- [ ] Rename all `*_step.py` files to drop the `_step` suffix +- [ ] Replace all `from motia import ...` with `from iii import ...` (and `from src.lib.iii_client import ...` for `iii` / `logger`) +- [ ] Convert every `config` dict + `handler` function to `iii.register_function(...)` + `iii.register_trigger(...)` calls +- [ ] Replace `ApiResponse(status=...)` with plain dicts using `statusCode` (e.g., `{"statusCode": 200, "body": ...}`) +- [ ] Replace `request.path_params` / `request.query_params` attribute access with dict-key access on the raw payload (`data["path_params"]`, `data.get("query_params")`) +- [ ] Replace `requestBody` with `request_body` in streaming handlers +- [ ] Replace `Stream` and `stateManager` usage with `iii.trigger()` calls or custom wrappers +- [ ] Migrate stream `on_join` / `on_leave` hooks to `register_function` + `register_trigger` (`stream:join` / `stream:leave`) +- [ ] Migrate `authenticate_stream` from `motia_config.py` to a registered function and set `auth_function` in `config.yaml` +- [ ] Delete `motia_config.py` +- [ ] Create `src/main.py` with side-effect imports for every handler module +- [ ] Replace `motia dev` with an `iii-exec` worker using `watch` + `exec` in your `config.yaml` +- [ ] Replace `motia build` with `pip install .` (and a `Dockerfile` for containerized deploys) +- [ ] Test all endpoints, cron jobs, queue subscribers, and stream handlers diff --git a/docs/0-13-0/changelog/index.mdx b/docs/0-13-0/changelog/index.mdx new file mode 100644 index 000000000..1819757cc --- /dev/null +++ b/docs/0-13-0/changelog/index.mdx @@ -0,0 +1,198 @@ +--- +title: "Changelog" +description: "Product updates and announcements for iii." +owner: "devrel" +type: "reference" +--- + + + ## Telemetry re-exports removed from public SDK surface + + **Breaking.** Convenience re-exports of OpenTelemetry accessors were dropped from the Rust, Node, Python, and browser SDKs. Underlying behavior is unchanged — only the public surface is smaller. Users who need a tracer or meter directly should depend on the OpenTelemetry library for their language. + + Removed symbols by language: + + | Symbol | Rust (`iii::*`) | Node (`iii-sdk/telemetry`) | Python (`iii.telemetry` / `iii.logger`) | Browser | + |---|---|---|---|---| + | `get_tracer` / `getTracer` | dropped (still at `iii::telemetry::get_tracer`) | dropped | renamed `_get_tracer` | already absent (asserted) | + | `get_meter` / `getMeter` | dropped (still at `iii::telemetry::get_meter`) | dropped | renamed `_get_meter` | already absent (asserted) | + | `is_initialized` | dropped (still at `iii::telemetry::is_initialized`) | n/a | renamed `_is_initialized` | already absent (asserted) | + | `SpanKind` | dropped (use `opentelemetry::trace::SpanKind`) | n/a | n/a | already absent (asserted) | + | `SpanStatus` / `SpanStatusCode` | dropped (use `opentelemetry::trace::Status`) | dropped | n/a | already absent (asserted) | + + ### Migration + + - For custom spans, prefer `withSpan` / `with_span` / `run_in_span`. These preserve trace context. + - To obtain a tracer or meter directly, depend on `@opentelemetry/api` (Node) or the `opentelemetry` crate / Python package and call its accessors. Rust users can also keep using `iii::telemetry::get_tracer` / `iii::telemetry::get_meter`. + + ```typescript + // Before (Node) + import { getTracer, getMeter, SpanStatusCode } from 'iii-sdk/telemetry' + + // After (Node) + import { trace, metrics, SpanStatusCode } from '@opentelemetry/api' + const tracer = trace.getTracer('my-service') + const meter = metrics.getMeter('my-service') + ``` + + ```rust + // Before (Rust) + use iii::{get_tracer, get_meter, SpanKind, SpanStatus}; + + // After (Rust) + use opentelemetry::global; + use opentelemetry::trace::{SpanKind, Status}; + let meter = global::meter("my-service"); + ``` + + ```python + # Before (Python) + from iii.telemetry import get_tracer, get_meter, is_initialized + + # After (Python) + from opentelemetry import trace, metrics + tracer = trace.get_tracer("my-service") + meter = metrics.get_meter("my-service") + ``` + + + + ## `iii sandbox` subcommand removed + + **Breaking.** The `iii sandbox` CLI subcommand is gone. Every sandbox operation now goes through `iii trigger`: + + ```bash + # before + iii sandbox create python --idle-timeout 300 + iii sandbox exec "$SB" -- python3 -c 'print(2+2)' + iii sandbox stop "$SB" + + # after + SB=$(iii trigger sandbox::create image=python idle_timeout_secs=300 | jq -r .sandbox_id) + iii trigger sandbox::exec sandbox_id="$SB" cmd=python3 args='["-c","print(2+2)"]' + iii trigger sandbox::stop sandbox_id="$SB" + ``` + + Each call also accepts a single `--json ''` payload (e.g. `iii trigger sandbox::exec --json '{"sandbox_id":"…","cmd":"python3","args":["-c","print(2+2)"]}'`), equivalent to the kv form shown above. + + `iii trigger` is request/response only, so the streaming flows the old subcommand offered (`exec` stdout/stderr stream, `upload`, `download`) are no longer available from the terminal. Use the SDK from worker code for those: `sandbox::exec` and `sandbox::fs::write` / `sandbox::fs::read` still expose the streaming channel. + + ## `iii trigger` reshape + + **Breaking.** `iii trigger` no longer accepts `--function-id` and `--payload`. The new form takes the function path as a positional argument and accepts payload fields as `key=value` tokens, an `--json ''` flag, or both: + + ```bash + # kv form + iii trigger orders::process amount=149.99 currency=USD + + # JSON form + iii trigger orders::process --json '{"amount": 149.99, "currency": "USD"}' + + # Combined: --json is the base, kv overrides individual keys + iii trigger orders::process --json '{"amount": 100}' amount=149.99 + ``` + + See [Triggers](/using-iii/triggers) for the full reference. + + ## `iii update --list-targets` + + `iii update` now exposes a `--list-targets` flag that prints every target accepted by `iii update ` (e.g. `self`, `console`, `worker`). Passing an unknown target now points users at this flag instead of failing silently. Rollback is not supported; reinstall a prior version manually with `curl -fsSL https://iii.dev/install.sh | sh -s -- --version `. + + + + ## Migrating from Motia + + **Breaking.** The Motia framework is deprecated in favor of using `iii-sdk` directly. Moving to the SDK unlocks multi-worker orchestration, browser connectivity via `iii-browser-sdk` with RBAC, and a direct understanding of iii's three primitives — Workers, Functions, and Triggers. Your existing Motia project becomes one worker in a larger iii deployment instead of a standalone monolith. + + [Node / TypeScript migration guide →](/changelog/0-11-0/migrating-from-motia-js) · [Python migration guide →](/changelog/0-11-0/migrating-from-motia-py) + + ## SDK discovery wrappers removed + + **Breaking.** The convenience discovery wrappers were removed from the Node, browser, Rust, and Python SDKs: + + - `listFunctions` / `list_functions` / `list_functions_async` + - `listWorkers` / `list_workers` / `list_workers_async` + - `listTriggers` / `list_triggers` / `list_triggers_async` + - `listTriggerTypes` / `list_trigger_types` / `list_trigger_types_async` + - `onFunctionsAvailable` / `on_functions_available` + + Discovery now goes through the core primitives directly: call `trigger()` against the built-in engine functions and register `engine::functions-available` like any other trigger type. This keeps the SDK surfaces aligned with the engine's "use the primitives directly" design. + + ## Worker RBAC + + The **iii-worker-manager** now supports role-based access control. Configure auth functions that validate WebSocket upgrade requests, attach per-session allow/deny lists for functions, control trigger registration, and auto-prefix function IDs for namespace isolation. An optional middleware function lets you intercept every invocation for audit logging, rate limiting, or payload enrichment. + + [Read the Worker RBAC guide →](/0-11-0/how-to/worker-rbac) + + ## Trigger format, validation, and metadata + + Trigger types now accept **`trigger_request_format`** and **`call_request_format`** fields (JSON Schema) so the engine can validate trigger configs and call payloads at registration time. Triggers also support an arbitrary **`metadata`** field for tagging and filtering. + + [Define request/response formats →](/0-11-0/how-to/define-request-response-formats) · [Trigger architecture →](/0-11-0/architecture/trigger-types) + + ## Browser SDK + + Your browser is now a first-class iii worker. The new `iii-browser-sdk` package connects to the engine over a single WebSocket and exposes the same core primitives as the Node SDK — `registerFunction`, `trigger`, `registerTrigger`, and `createChannel` all work identically. Build real-time dashboards, collaborative apps, and bi-directional frontends without REST endpoints or polling. + + [Use iii in the browser →](/0-11-0/how-to/use-iii-in-the-browser) + + ## Sandbox and Container Workers + + Workers can now run as **container workers** or **sandbox workers**. Container workers are OCI images managed through the `iii worker` CLI — add an image, configure it in `config.yaml`, and the engine pulls, extracts, and runs it in an isolated sandbox. For local development, `iii worker add ./my-project` registers a local directory as a first-class managed worker that runs inside a lightweight microVM with auto-detected runtimes, dependency caching, and full lifecycle support (`start`, `stop`, `list`, `remove`) — no Dockerfiles needed. Requires macOS Apple Silicon or Linux with KVM. + + [Managing Container Workers →](/0-11-0/how-to/managing-container-workers) · [Developing Sandbox Workers →](/0-11-0/how-to/developing-sandbox-workers) + + ## `iii worker exec` + + A new `iii worker exec -- ` command runs arbitrary commands inside a running worker's microVM — think `docker exec` for iii workers. stdin/stdout/stderr flow through, exit codes pass back, Ctrl-C delivers SIGINT (twice for SIGKILL). TTY mode auto-detects when both stdin and stdout are terminals, so `iii worker exec my-worker -- sh` in a terminal gives you a real interactive shell with line editing and job control. Pass `--timeout 30s` to bound runaway commands (exit 124 matches coreutils). + + [Exec into a running worker →](/0-11-0/how-to/managing-container-workers#5-exec-into-a-running-worker) + + ## Reproducible worker installs + + Registry-managed workers can now be pinned in `iii.lock`. `iii worker add` writes the resolved worker graph when the registry provides one, binary workers can record artifacts for multiple platform targets, `iii worker verify` checks that `config.yaml` is represented in the lockfile, and `iii worker update [worker]` refreshes locked pins intentionally. + + [Reproduce Worker Installs →](/0-11-0/how-to/reproduce-worker-installs) + + ## Topic-based fan-out queues + + **Breaking.** The topic-based queue API has been renamed. The trigger type changes from `queue` to `durable:subscriber`, and the publish function changes from `enqueue` to `iii::durable::publish`: + + ```typescript + // Before + registerTrigger({ type: 'queue', function_id: 'my::handler', config: { topic: 'order.created' } }) + trigger({ function_id: 'enqueue', payload: { topic: 'order.created', data } }) + + // After + registerTrigger({ type: 'durable:subscriber', function_id: 'my::handler', config: { topic: 'order.created' } }) + trigger({ function_id: 'iii::durable::publish', payload: { topic: 'order.created', data } }) + ``` + + Messages now fan out to every subscriber, with each function processing its copy independently and retrying on its own schedule. If a function has multiple replicas, they compete on a shared per-function queue. An optional `condition_function_id` lets you filter messages server-side before they reach the handler. + + [Use topic-based queues →](/0-11-0/how-to/use-topic-queues) + + ## Node SDK: `registerFunction` signature change + + **Breaking.** The `registerFunction` API now takes the function ID as a plain string instead of an options object: + + ```typescript + // Before + registerFunction({ id: 'function-id' }, handler) + + // After + registerFunction('function-id', handler, {}) + ``` + + The options object (metadata, request/response formats) moves to an optional third argument. + + ## Everything is a worker + + **Breaking.** We simplified iii down to three primitives: **Workers**, **Functions**, and **Triggers**. Modules were always workers in disguise -- they connect to the engine, register functions, and react to triggers just like SDK workers do. Now the naming reflects that. + + - **Config YAML** — `modules:` top-level key renamed to `workers:`, `class:` field renamed to `name:` with short identifiers. + - **Rust API** — `Module` trait → `Worker`, `register_module!` → `register_worker!`, `EngineBuilder::add_module()` → `add_worker()`. + - **Adapter IDs** — changed from long Rust-style paths to short names: `kv`, `redis`, `builtin`, `rabbitmq`, `local`, `bridge`. + + [Read the full story and migration guide →](/changelog/0-11-0/everything-is-a-worker) + diff --git a/docs/0-13-0/creating-workers/functions.mdx b/docs/0-13-0/creating-workers/functions.mdx new file mode 100644 index 000000000..9605fb99c --- /dev/null +++ b/docs/0-13-0/creating-workers/functions.mdx @@ -0,0 +1,204 @@ +--- +title: "Functions" +description: "Author the functions a new worker contributes to an iii system." +owner: "devrel" +type: "how-to" +--- + +## What "writing a function" means + +A worker contributes capability by registering functions. Each function has an `id` of the form +`service::name`, a handler that receives the payload and returns a result, and optional JSON +Schemas that describe the request and response shape. + +For how callers invoke functions (`worker.trigger` / `iii trigger` / event-bound triggers), see +[Using iii / Functions](/using-iii/functions) and +[Using iii / Triggers](/using-iii/triggers). This page is about the authoring surface. + +## Register a function + +Inside the worker, register the function with the SDK. The `id` is what callers pass as +`function_id`; the handler signature is the same regardless of how the invocation arrived (direct +call, HTTP trigger, cron, queue message). + + + + ```typescript + import { registerWorker } from "iii-sdk"; + + const worker = registerWorker(process.env.III_URL); + + worker.registerFunction("math::add", async (payload: { a: number; b: number }) => { + return { c: payload.a + payload.b }; + }); + ``` + + + ```python + import os + from iii import register_worker, InitOptions + + worker = register_worker( + os.environ.get("III_URL"), + InitOptions(worker_name="math-worker"), + ) + + def add_handler(payload: dict) -> dict: + return {"c": payload["a"] + payload["b"]} + + worker.register_function("math::add", add_handler) + ``` + + + ```rust + use iii_sdk::{InitOptions, RegisterFunction, register_worker}; + + let url = std::env::var("III_URL").expect("III_URL must be set"); + let worker = register_worker(&url, InitOptions::default()); + + worker.register_function(RegisterFunction::new("math::add", |input: AddInput| { + Ok(serde_json::json!({ "c": input.a + input.b })) + }))?; + ``` + + + +## Attach request and response schemas + +Attach JSON Schemas to the registration so the request and response shape are documented alongside +the function. The schemas are stored with the function and surface in the iii console and the +agent-readable skills. + + + Runtime validation is not yet supported. Attached schemas are metadata only; the engine does not + reject payloads or handler return values that don't match them. + + + + + ```typescript + import { registerWorker } from "iii-sdk"; + + const worker = registerWorker(process.env.III_URL); + + worker.registerFunction( + "math::add", + async (payload) => ({ c: payload.a + payload.b }), + { + request_schema: { + type: "object", + properties: { a: { type: "number" }, b: { type: "number" } }, + required: ["a", "b"], + }, + response_schema: { + type: "object", + properties: { c: { type: "number" } }, + required: ["c"], + }, + }, + ); + ``` + + + ```python + import os + from iii import register_worker, InitOptions + + worker = register_worker( + os.environ.get("III_URL"), + InitOptions(worker_name="math-worker"), + ) + + worker.register_function( + "math::add", + add_handler, + request_schema={ + "type": "object", + "properties": {"a": {"type": "number"}, "b": {"type": "number"}}, + "required": ["a", "b"], + }, + response_schema={ + "type": "object", + "properties": {"c": {"type": "number"}}, + "required": ["c"], + }, + ) + ``` + + + ```rust + use iii_sdk::{InitOptions, RegisterFunction, register_worker}; + use serde_json::json; + + let url = std::env::var("III_URL").expect("III_URL must be set"); + let worker = register_worker(&url, InitOptions::default()); + + worker.register_function( + RegisterFunction::new("math::add", |input: AddInput| { + Ok(serde_json::json!({ "c": input.a + input.b })) + }) + .request_schema(json!({ + "type": "object", + "properties": { "a": { "type": "number" }, "b": { "type": "number" } }, + "required": ["a", "b"], + })) + .response_schema(json!({ + "type": "object", + "properties": { "c": { "type": "number" } }, + "required": ["c"], + })), + )?; + ``` + + + +The schemas also feed the iii console and the agent-readable skills. + +## Return values and errors + +A function returns either a value (which the handler is responsible for shaping to match its +documented response schema) or an error. Errors raised inside the handler are propagated to the +caller as invocation errors with the worker's stack trace; the engine doesn't swallow them. Use +this distinction to express expected failures (return a structured error value) versus unexpected +ones (throw / raise / return `Err`). + +## Unregister a function + +`registerFunction` returns a handle with an `unregister()` method that removes the function from +the engine at runtime. When the worker disconnects, all of its functions are removed automatically +and pending invocations error out. + + + + ```typescript + const add = worker.registerFunction("math::add", async (payload) => { + return { c: payload.a + payload.b }; + }); + + add.unregister(); + ``` + + + ```python + add = worker.register_function("math::add", add_handler) + + add.unregister() + ``` + + + ```rust + let add = worker.register_function(RegisterFunction::new("math::add", |input: AddInput| { + Ok(serde_json::json!({ "c": input.a + input.b })) + })); + + add.unregister(); + ``` + + + +## What goes in Worker Docs + +The function ids your worker exposes, what each one does, and any worker-specific semantics +(idempotency, rate limits, side effects) belong in this worker's Worker Docs. Keep iii-level +concepts (the registration surface, schema metadata, error propagation) here; document the +per-function specifics there. diff --git a/docs/0-13-0/creating-workers/functions.mdx.skill.md b/docs/0-13-0/creating-workers/functions.mdx.skill.md new file mode 100644 index 000000000..f9ff86224 --- /dev/null +++ b/docs/0-13-0/creating-workers/functions.mdx.skill.md @@ -0,0 +1,202 @@ + + +# Functions + + +## What "writing a function" means + +A worker contributes capability by registering functions. Each function has an `id` of the form +`service::name`, a handler that receives the payload and returns a result, and optional JSON +Schemas that describe the request and response shape. + +For how callers invoke functions (`worker.trigger` / `iii trigger` / event-bound triggers), see +[Using iii / Functions](/using-iii/functions) and +[Using iii / Triggers](/using-iii/triggers). This page is about the authoring surface. + +## Register a function + +Inside the worker, register the function with the SDK. The `id` is what callers pass as +`function_id`; the handler signature is the same regardless of how the invocation arrived (direct +call, HTTP trigger, cron, queue message). + + + + ```typescript + import { registerWorker } from "iii-sdk"; + + const worker = registerWorker(process.env.III_URL); + + worker.registerFunction("math::add", async (payload: { a: number; b: number }) => { + return { c: payload.a + payload.b }; + }); + ``` + + + ```python + import os + from iii import register_worker, InitOptions + + worker = register_worker( + os.environ.get("III_URL"), + InitOptions(worker_name="math-worker"), + ) + + def add_handler(payload: dict) -> dict: + return {"c": payload["a"] + payload["b"]} + + worker.register_function("math::add", add_handler) + ``` + + + ```rust + use iii_sdk::{InitOptions, RegisterFunction, register_worker}; + + let url = std::env::var("III_URL").expect("III_URL must be set"); + let worker = register_worker(&url, InitOptions::default()); + + worker.register_function(RegisterFunction::new("math::add", |input: AddInput| { + Ok(serde_json::json!({ "c": input.a + input.b })) + }))?; + ``` + + + +## Attach request and response schemas + +Attach JSON Schemas to the registration so the request and response shape are documented alongside +the function. The schemas are stored with the function and surface in the iii console and the +agent-readable skills. + + + Runtime validation is not yet supported. Attached schemas are metadata only; the engine does not + reject payloads or handler return values that don't match them. + + + + + ```typescript + import { registerWorker } from "iii-sdk"; + + const worker = registerWorker(process.env.III_URL); + + worker.registerFunction( + "math::add", + async (payload) => ({ c: payload.a + payload.b }), + { + request_schema: { + type: "object", + properties: { a: { type: "number" }, b: { type: "number" } }, + required: ["a", "b"], + }, + response_schema: { + type: "object", + properties: { c: { type: "number" } }, + required: ["c"], + }, + }, + ); + ``` + + + ```python + import os + from iii import register_worker, InitOptions + + worker = register_worker( + os.environ.get("III_URL"), + InitOptions(worker_name="math-worker"), + ) + + worker.register_function( + "math::add", + add_handler, + request_schema={ + "type": "object", + "properties": {"a": {"type": "number"}, "b": {"type": "number"}}, + "required": ["a", "b"], + }, + response_schema={ + "type": "object", + "properties": {"c": {"type": "number"}}, + "required": ["c"], + }, + ) + ``` + + + ```rust + use iii_sdk::{InitOptions, RegisterFunction, register_worker}; + use serde_json::json; + + let url = std::env::var("III_URL").expect("III_URL must be set"); + let worker = register_worker(&url, InitOptions::default()); + + worker.register_function( + RegisterFunction::new("math::add", |input: AddInput| { + Ok(serde_json::json!({ "c": input.a + input.b })) + }) + .request_schema(json!({ + "type": "object", + "properties": { "a": { "type": "number" }, "b": { "type": "number" } }, + "required": ["a", "b"], + })) + .response_schema(json!({ + "type": "object", + "properties": { "c": { "type": "number" } }, + "required": ["c"], + })), + )?; + ``` + + + +The schemas also feed the iii console and the agent-readable skills. + +## Return values and errors + +A function returns either a value (which the handler is responsible for shaping to match its +documented response schema) or an error. Errors raised inside the handler are propagated to the +caller as invocation errors with the worker's stack trace; the engine doesn't swallow them. Use +this distinction to express expected failures (return a structured error value) versus unexpected +ones (throw / raise / return `Err`). + +## Unregister a function + +`registerFunction` returns a handle with an `unregister()` method that removes the function from +the engine at runtime. When the worker disconnects, all of its functions are removed automatically +and pending invocations error out. + + + + ```typescript + const add = worker.registerFunction("math::add", async (payload) => { + return { c: payload.a + payload.b }; + }); + + add.unregister(); + ``` + + + ```python + add = worker.register_function("math::add", add_handler) + + add.unregister() + ``` + + + ```rust + let add = worker.register_function(RegisterFunction::new("math::add", |input: AddInput| { + Ok(serde_json::json!({ "c": input.a + input.b })) + })); + + add.unregister(); + ``` + + + +## What goes in Worker Docs + +The function ids your worker exposes, what each one does, and any worker-specific semantics +(idempotency, rate limits, side effects) belong in this worker's Worker Docs. Keep iii-level +concepts (the registration surface, schema metadata, error propagation) here; document the +per-function specifics there. diff --git a/docs/0-13-0/creating-workers/triggers.mdx b/docs/0-13-0/creating-workers/triggers.mdx new file mode 100644 index 000000000..5f35f384c --- /dev/null +++ b/docs/0-13-0/creating-workers/triggers.mdx @@ -0,0 +1,201 @@ +--- +title: "Triggers" +description: "Author new trigger types so other workers' functions can run on the events your worker emits." +owner: "devrel" +type: "how-to" +--- + +## What "writing a trigger" means + +Workers don't only call functions; they can also _source_ events. When a worker advertises a +trigger type (e.g. `webhook`, a custom schedule, or any external event source you implement), any +other worker can bind its functions to that type. The engine routes binding requests to your +trigger type's handler, and your worker invokes the bound functions when the underlying event +fires. + +This page is about authoring new trigger types from inside your worker. For invoking functions and +binding triggers to functions from the consumer side, see [Using iii / Triggers](/using-iii/triggers). + +## Declare a trigger type + +Inside the worker, register the trigger type once during startup. You pass an `id` and +`description` plus a handler with `registerTrigger` / `unregisterTrigger` callbacks. The engine +calls these whenever a consumer binds or unbinds a function against your trigger type, so your +worker can keep its own table of `{ trigger id → function id, config }` bindings. + +`registerTriggerType` returns a typed handle. Use it later to bind functions you own to this +trigger type, or to tear the type down. + + + + ```typescript + import { registerWorker } from "iii-sdk"; + import type { TriggerConfig, TriggerHandler } from "iii-sdk"; + + const worker = registerWorker(process.env.III_URL); + + type WebhookTriggerConfig = { + path: string; + secret?: string; + }; + + const webhookHandler: TriggerHandler = { + async registerTrigger(config: TriggerConfig) { + // Stash { config.id, config.function_id, config.config } so the + // worker's HTTP listener knows which function to invoke when a + // request matches `config.config.path`. + }, + async unregisterTrigger(config: TriggerConfig) { + // Remove the binding from the worker's table. + }, + }; + + const webhook = worker.registerTriggerType( + { id: "webhook", description: "Incoming webhook trigger" }, + webhookHandler, + ); + ``` + + + ```python + import os + from iii import ( + InitOptions, + RegisterTriggerTypeInput, + TriggerConfig, + TriggerHandler, + register_worker, + ) + + worker = register_worker( + os.environ.get("III_URL"), + InitOptions(worker_name="webhook-worker"), + ) + + class WebhookHandler(TriggerHandler): + async def register_trigger(self, config: TriggerConfig) -> None: + # Stash config.id, config.function_id, config.config + ... + + async def unregister_trigger(self, config: TriggerConfig) -> None: + ... + + webhook = worker.register_trigger_type( + RegisterTriggerTypeInput(id="webhook", description="Incoming webhook trigger"), + WebhookHandler(), + ) + ``` + + + ```rust + use iii_sdk::{ + InitOptions, RegisterTriggerType, TriggerConfig, TriggerHandler, register_worker, + }; + + let url = std::env::var("III_URL").expect("III_URL must be set"); + let worker = register_worker(&url, InitOptions::default()); + + struct WebhookHandler; + + #[async_trait::async_trait] + impl TriggerHandler for WebhookHandler { + async fn register_trigger(&self, config: TriggerConfig) -> Result<(), iii_sdk::IIIError> { + // Stash config.id, config.function_id, config.config + Ok(()) + } + + async fn unregister_trigger(&self, config: TriggerConfig) -> Result<(), iii_sdk::IIIError> { + Ok(()) + } + } + + let webhook = worker.register_trigger_type( + RegisterTriggerType::new("webhook", "Incoming webhook trigger", WebhookHandler), + ); + ``` + + + +For typed `config` and call-payload schemas, attach Pydantic, Zod, or `schemars::JsonSchema` types +to the registration in your language of choice. See each SDK's reference for the exact builder. + +## Dispatch events to bound functions + +There is no special "fire" API. When the underlying event source delivers something (an incoming +HTTP request, a cron tick, a webhook hit), your worker looks up the bindings it stashed in its +`registerTrigger` callback and invokes each bound function via `worker.trigger(...)`. + + + + ```typescript + // Inside the worker's HTTP listener, after matching a request to a binding: + await worker.trigger({ + function_id: binding.function_id, + payload: { method, headers, body }, + }); + ``` + + + ```python + # Inside the worker's HTTP listener, after matching a request to a binding: + worker.trigger({ + "function_id": binding["function_id"], + "payload": {"method": method, "headers": headers, "body": body}, + }) + ``` + + + ```rust + use iii_sdk::TriggerRequest; + use serde_json::json; + + // Inside the worker's HTTP listener, after matching a request to a binding: + worker + .trigger(TriggerRequest { + function_id: binding.function_id.clone(), + payload: json!({ "method": method, "headers": headers, "body": body }), + action: None, + timeout_ms: None, + }) + .await?; + ``` + + + +On each dispatched event, the engine evaluates the consumer's `config` and optional +`condition_function_id`, then routes matching invocations to the bound function and returns the +result to the caller. + +## Unregister a trigger type + +`registerTriggerType` returns a handle with an `unregister()` method that tears down the trigger +type at runtime. When the worker disconnects, all trigger types it advertised are removed +automatically and the engine stops routing events that depended on them. + + + + ```typescript + webhook.unregister(); + ``` + + + ```python + worker.unregister_trigger_type( + RegisterTriggerTypeInput(id="webhook", description="Incoming webhook trigger"), + ) + ``` + + + ```rust + worker.unregister_trigger_type("webhook"); + ``` + + + +## What goes in Worker Docs + +The trigger type _id_, the shape of the `config` consumers pass when binding, the call-payload +shape your worker delivers to bound functions, the event ordering guarantees, and any +back-pressure or retry semantics belong in this worker's Worker Docs so consumers know what to +pass when they call `registerTrigger`. Keep iii-level concepts (the trigger binding model itself, +condition gates) here; document the per-type specifics there. diff --git a/docs/0-13-0/creating-workers/triggers.mdx.skill.md b/docs/0-13-0/creating-workers/triggers.mdx.skill.md new file mode 100644 index 000000000..45a2c31af --- /dev/null +++ b/docs/0-13-0/creating-workers/triggers.mdx.skill.md @@ -0,0 +1,197 @@ + + + +## What "writing a trigger" means + +Workers don't only call functions; they can also _source_ events. When a worker advertises a +trigger type (e.g. `webhook`, a custom schedule, or any external event source you implement), any +other worker can bind its functions to that type. The engine routes binding requests to your +trigger type's handler, and your worker invokes the bound functions when the underlying event +fires. + +This page is about authoring new trigger types from inside your worker. For invoking functions and +binding triggers to functions from the consumer side, see [Using iii / Triggers](/using-iii/triggers). + +## Declare a trigger type + +Inside the worker, register the trigger type once during startup. You pass an `id` and +`description` plus a handler with `registerTrigger` / `unregisterTrigger` callbacks. The engine +calls these whenever a consumer binds or unbinds a function against your trigger type, so your +worker can keep its own table of `{ trigger id → function id, config }` bindings. + +`registerTriggerType` returns a typed handle. Use it later to bind functions you own to this +trigger type, or to tear the type down. + + + + ```typescript + import { registerWorker } from "iii-sdk"; + import type { TriggerConfig, TriggerHandler } from "iii-sdk"; + + const worker = registerWorker(process.env.III_URL); + + type WebhookTriggerConfig = { + path: string; + secret?: string; + }; + + const webhookHandler: TriggerHandler = { + async registerTrigger(config: TriggerConfig) { + // Stash { config.id, config.function_id, config.config } so the + // worker's HTTP listener knows which function to invoke when a + // request matches `config.config.path`. + }, + async unregisterTrigger(config: TriggerConfig) { + // Remove the binding from the worker's table. + }, + }; + + const webhook = worker.registerTriggerType( + { id: "webhook", description: "Incoming webhook trigger" }, + webhookHandler, + ); + ``` + + + ```python + import os + from iii import ( + InitOptions, + RegisterTriggerTypeInput, + TriggerConfig, + TriggerHandler, + register_worker, + ) + + worker = register_worker( + os.environ.get("III_URL"), + InitOptions(worker_name="webhook-worker"), + ) + + class WebhookHandler(TriggerHandler): + async def register_trigger(self, config: TriggerConfig) -> None: + # Stash config.id, config.function_id, config.config + ... + + async def unregister_trigger(self, config: TriggerConfig) -> None: + ... + + webhook = worker.register_trigger_type( + RegisterTriggerTypeInput(id="webhook", description="Incoming webhook trigger"), + WebhookHandler(), + ) + ``` + + + ```rust + use iii_sdk::{ + InitOptions, RegisterTriggerType, TriggerConfig, TriggerHandler, register_worker, + }; + + let url = std::env::var("III_URL").expect("III_URL must be set"); + let worker = register_worker(&url, InitOptions::default()); + + struct WebhookHandler; + + #[async_trait::async_trait] + impl TriggerHandler for WebhookHandler { + async fn register_trigger(&self, config: TriggerConfig) -> Result<(), iii_sdk::IIIError> { + // Stash config.id, config.function_id, config.config + Ok(()) + } + + async fn unregister_trigger(&self, config: TriggerConfig) -> Result<(), iii_sdk::IIIError> { + Ok(()) + } + } + + let webhook = worker.register_trigger_type( + RegisterTriggerType::new("webhook", "Incoming webhook trigger", WebhookHandler), + ); + ``` + + + +For typed `config` and call-payload schemas, attach Pydantic, Zod, or `schemars::JsonSchema` types +to the registration in your language of choice. See each SDK's reference for the exact builder. + +## Dispatch events to bound functions + +There is no special "fire" API. When the underlying event source delivers something (an incoming +HTTP request, a cron tick, a webhook hit), your worker looks up the bindings it stashed in its +`registerTrigger` callback and invokes each bound function via `worker.trigger(...)`. + + + + ```typescript + // Inside the worker's HTTP listener, after matching a request to a binding: + await worker.trigger({ + function_id: binding.function_id, + payload: { method, headers, body }, + }); + ``` + + + ```python + # Inside the worker's HTTP listener, after matching a request to a binding: + worker.trigger({ + "function_id": binding["function_id"], + "payload": {"method": method, "headers": headers, "body": body}, + }) + ``` + + + ```rust + use iii_sdk::TriggerRequest; + use serde_json::json; + + // Inside the worker's HTTP listener, after matching a request to a binding: + worker + .trigger(TriggerRequest { + function_id: binding.function_id.clone(), + payload: json!({ "method": method, "headers": headers, "body": body }), + action: None, + timeout_ms: None, + }) + .await?; + ``` + + + +On each dispatched event, the engine evaluates the consumer's `config` and optional +`condition_function_id`, then routes matching invocations to the bound function and returns the +result to the caller. + +## Unregister a trigger type + +`registerTriggerType` returns a handle with an `unregister()` method that tears down the trigger +type at runtime. When the worker disconnects, all trigger types it advertised are removed +automatically and the engine stops routing events that depended on them. + + + + ```typescript + webhook.unregister(); + ``` + + + ```python + worker.unregister_trigger_type( + RegisterTriggerTypeInput(id="webhook", description="Incoming webhook trigger"), + ) + ``` + + + ```rust + worker.unregister_trigger_type("webhook"); + ``` + + + +## What goes in Worker Docs + +The trigger type _id_, the shape of the `config` consumers pass when binding, the call-payload +shape your worker delivers to bound functions, the event ordering guarantees, and any +back-pressure or retry semantics belong in this worker's Worker Docs so consumers know what to +pass when they call `registerTrigger`. Keep iii-level concepts (the trigger binding model itself, +condition gates) here; document the per-type specifics there. diff --git a/docs/0-13-0/creating-workers/workers-registry.mdx b/docs/0-13-0/creating-workers/workers-registry.mdx new file mode 100644 index 000000000..302cd6e71 --- /dev/null +++ b/docs/0-13-0/creating-workers/workers-registry.mdx @@ -0,0 +1,145 @@ +--- +title: "Registry" +description: "Publishing your workers to the iii registry." +owner: "devrel" +type: "how-to" +--- + +{/* Note: move this up */} + +The iii registry at [workers.iii.dev](https://workers.iii.dev/) is where published workers live so +other iii projects can install them with `iii worker add `. + +## Publish a worker + +Publishing a worker uploads its binary or OCI image to the registry, records its semver version, and +makes the worker installable by name from any iii project. + +{/* TODO: capture the canonical publish command (likely `iii worker publish` or similar), the authentication requirements, and the metadata the registry expects (description, repo URL, supported platforms, etc.). */} + +## Version your worker + +Workers in the registry follow semver. Patch bumps for bug fixes, minor bumps for additive +capability, major bumps for breaking changes to function or trigger signatures. + +{/* TODO: document how versions are tagged in the worker repo (git tag pattern), how the publish command resolves the version, and how to publish pre-releases. */} + +## Build binary artifacts for multiple platforms + +Binary workers can publish artifacts for multiple platform targets in a single registry entry (macOS +arm64/x64, Linux arm64/x64/armv7, Windows arm64/x64/x86). One published version covers every +supported host without separate publications per platform. + +{/* TODO: document the cross-build flow, the supported target triples, how the artifacts are signed/checksummed, and where they're uploaded. */} + +## Update or remove a published worker + +{/* TODO: cover how to publish a new version (semver bump + republish), how to deprecate a worker, and whether/how a published version can be retracted (yanked). */} + +## Bundle workers (tar.gz archives) + +Bundle workers are a third artifact kind alongside `binary` and `image`. The registry serves a +single `tar.gz` archive that contains the worker's bundled source plus an `iii.worker.yaml` +manifest at the archive root. `iii worker add ` downloads, verifies a SHA-256 checksum, +extracts the archive into `~/.iii/workers-bundle//`, and runs it through the existing +libkrun rails (the same sandbox path used by local-path workers, minus the host-side source +watcher). + +Use a bundle when: + +- You ship a pre-built JavaScript bundle (`esbuild`, `tsdown`, `bun build`) or a packaged Python + worker and don't want to publish a Docker image. +- You want artifacts measured in KB, not MB. Only the bundled source travels in the archive; + the runtime ships with the engine-allowlisted base image (`docker.io/iiidev/node:latest` or + `docker.io/iiidev/python:latest`). +- You want install to look identical to other registry workers from the user's perspective + (`iii worker add my-worker`, same as binary and OCI). + +### Registry response shape + +```json +{ + "type": "bundle", + "name": "my-worker", + "version": "1.2.0", + "archive_url": "https://cdn.workers.iii.dev/my-worker/1.2.0/bundle.tar.gz", + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" +} +``` + +The engine GETs `archive_url`, streams the bytes through a SHA-256 hasher, and compares against +`sha256`. Mismatches abort the install and delete the downloaded blob immediately. + +### Archive layout + +The archive root MUST contain `iii.worker.yaml`. Anything else sits at runtime-discoverable +paths from the bundle's perspective. + +```text +my-worker-1.2.0.tar.gz +├── iii.worker.yaml +├── bundle.js +└── assets/ + └── ... +``` + +### Manifest contract (`iii.worker.yaml`) + +Bundle manifests use a strict subset of the local-worker manifest. Three fields are explicitly +**rejected**: + +- `scripts.setup`: would execute publisher-supplied shell during install (a supply-chain + smuggling vector). +- `scripts.install`: same reason. Vendor dependencies into the bundle instead. +- `runtime.base_image`: would let a bundle pull an arbitrary OCI image as its rootfs. Bundles + use the engine-allowlisted base image instead. + +Required fields: + +- `name`: must equal the install target (the value passed to `iii worker add`). +- `scripts.start`: a non-empty shell string. The engine `exec`s this inside the sandbox VM. + Example: `node bundle.js`, `python -m worker`, `bun run bundle.js`. + +Optional fields (clamped against engine caps, with a `W182 BundleResourceClamped` warning when +the request exceeds the cap): + +- `resources.cpus`: defaults to `2`, clamped to `4`. +- `resources.memory`: defaults to `2048` MiB, clamped to `4096` MiB. + +```yaml +name: my-worker +version: 1.2.0 +scripts: + start: node bundle.js +resources: + cpus: 2 + memory: 2048 +``` + +### Archive safety policy + +Bundle archives are extracted with tighter limits than OCI layers: + +| Limit | Value | +|-----------------------------|--------------------| +| Total uncompressed size | 64 MiB | +| Largest single file | 32 MiB | +| Maximum entry count | 1024 | +| Maximum directory depth | 16 | +| Allowed tar entry types | Regular, Directory | + +Archives containing symlinks, hard links, character devices, FIFOs, or paths with `..` +components are rejected with `W181 BundleArchiveUnsafe`. + +### Error codes + +| Code | Failure | +|--------|----------------------------------------------------------------------------------------------------| +| `W142` | Archive download failed (HTTP error, unexpected content-type, size cap, sha256 mismatch). | +| `W180` | Manifest rejected (forbidden field like `scripts.setup` or `runtime.base_image`). | +| `W181` | Archive contains unsafe entries (symlink, hardlink, traversal, oversized, too many entries). | +| `W182` | Resource request exceeded engine cap; install proceeded with clamped values (warn, not fail). | +| `W183` | Dependency graph too wide or too deep (max depth 5, max transitive count 32). | + +{/* TODO: document the publish flow (`iii worker publish bundle.tar.gz`?), the registry's +storage layout, and the recommended bundler configurations for Node/Bun/Python. */} diff --git a/docs/0-13-0/creating-workers/workers-registry.mdx.skill.md b/docs/0-13-0/creating-workers/workers-registry.mdx.skill.md new file mode 100644 index 000000000..35c90fd49 --- /dev/null +++ b/docs/0-13-0/creating-workers/workers-registry.mdx.skill.md @@ -0,0 +1,143 @@ + + +# Registry + + +{/* Note: move this up */} + +The iii registry at [workers.iii.dev](https://workers.iii.dev/) is where published workers live so +other iii projects can install them with `iii worker add `. + +## Publish a worker + +Publishing a worker uploads its binary or OCI image to the registry, records its semver version, and +makes the worker installable by name from any iii project. + +{/* TODO: capture the canonical publish command (likely `iii worker publish` or similar), the authentication requirements, and the metadata the registry expects (description, repo URL, supported platforms, etc.). */} + +## Version your worker + +Workers in the registry follow semver. Patch bumps for bug fixes, minor bumps for additive +capability, major bumps for breaking changes to function or trigger signatures. + +{/* TODO: document how versions are tagged in the worker repo (git tag pattern), how the publish command resolves the version, and how to publish pre-releases. */} + +## Build binary artifacts for multiple platforms + +Binary workers can publish artifacts for multiple platform targets in a single registry entry (macOS +arm64/x64, Linux arm64/x64/armv7, Windows arm64/x64/x86). One published version covers every +supported host without separate publications per platform. + +{/* TODO: document the cross-build flow, the supported target triples, how the artifacts are signed/checksummed, and where they're uploaded. */} + +## Update or remove a published worker + +{/* TODO: cover how to publish a new version (semver bump + republish), how to deprecate a worker, and whether/how a published version can be retracted (yanked). */} + +## Bundle workers (tar.gz archives) + +Bundle workers are a third artifact kind alongside `binary` and `image`. The registry serves a +single `tar.gz` archive that contains the worker's bundled source plus an `iii.worker.yaml` +manifest at the archive root. `iii worker add ` downloads, verifies a SHA-256 checksum, +extracts the archive into `~/.iii/workers-bundle//`, and runs it through the existing +libkrun rails (the same sandbox path used by local-path workers, minus the host-side source +watcher). + +Use a bundle when: + +- You ship a pre-built JavaScript bundle (`esbuild`, `tsdown`, `bun build`) or a packaged Python + worker and don't want to publish a Docker image. +- You want artifacts measured in KB, not MB. Only the bundled source travels in the archive; + the runtime ships with the engine-allowlisted base image (`docker.io/iiidev/node:latest` or + `docker.io/iiidev/python:latest`). +- You want install to look identical to other registry workers from the user's perspective + (`iii worker add my-worker`, same as binary and OCI). + +### Registry response shape + +```json +{ + "type": "bundle", + "name": "my-worker", + "version": "1.2.0", + "archive_url": "https://cdn.workers.iii.dev/my-worker/1.2.0/bundle.tar.gz", + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" +} +``` + +The engine GETs `archive_url`, streams the bytes through a SHA-256 hasher, and compares against +`sha256`. Mismatches abort the install and delete the downloaded blob immediately. + +### Archive layout + +The archive root MUST contain `iii.worker.yaml`. Anything else sits at runtime-discoverable +paths from the bundle's perspective. + +```text +my-worker-1.2.0.tar.gz +├── iii.worker.yaml +├── bundle.js +└── assets/ + └── ... +``` + +### Manifest contract (`iii.worker.yaml`) + +Bundle manifests use a strict subset of the local-worker manifest. Three fields are explicitly +**rejected**: + +- `scripts.setup`: would execute publisher-supplied shell during install (a supply-chain + smuggling vector). +- `scripts.install`: same reason. Vendor dependencies into the bundle instead. +- `runtime.base_image`: would let a bundle pull an arbitrary OCI image as its rootfs. Bundles + use the engine-allowlisted base image instead. + +Required fields: + +- `name`: must equal the install target (the value passed to `iii worker add`). +- `scripts.start`: a non-empty shell string. The engine `exec`s this inside the sandbox VM. + Example: `node bundle.js`, `python -m worker`, `bun run bundle.js`. + +Optional fields (clamped against engine caps, with a `W182 BundleResourceClamped` warning when +the request exceeds the cap): + +- `resources.cpus`: defaults to `2`, clamped to `4`. +- `resources.memory`: defaults to `2048` MiB, clamped to `4096` MiB. + +```yaml +name: my-worker +version: 1.2.0 +scripts: + start: node bundle.js +resources: + cpus: 2 + memory: 2048 +``` + +### Archive safety policy + +Bundle archives are extracted with tighter limits than OCI layers: + +| Limit | Value | +|-----------------------------|--------------------| +| Total uncompressed size | 64 MiB | +| Largest single file | 32 MiB | +| Maximum entry count | 1024 | +| Maximum directory depth | 16 | +| Allowed tar entry types | Regular, Directory | + +Archives containing symlinks, hard links, character devices, FIFOs, or paths with `..` +components are rejected with `W181 BundleArchiveUnsafe`. + +### Error codes + +| Code | Failure | +|--------|----------------------------------------------------------------------------------------------------| +| `W142` | Archive download failed (HTTP error, unexpected content-type, size cap, sha256 mismatch). | +| `W180` | Manifest rejected (forbidden field like `scripts.setup` or `runtime.base_image`). | +| `W181` | Archive contains unsafe entries (symlink, hardlink, traversal, oversized, too many entries). | +| `W182` | Resource request exceeded engine cap; install proceeded with clamped values (warn, not fail). | +| `W183` | Dependency graph too wide or too deep (max depth 5, max transitive count 32). | + +{/* TODO: document the publish flow (`iii worker publish bundle.tar.gz`?), the registry's +storage layout, and the recommended bundler configurations for Node/Bun/Python. */} diff --git a/docs/0-13-0/creating-workers/workers.mdx b/docs/0-13-0/creating-workers/workers.mdx new file mode 100644 index 000000000..72305cb34 --- /dev/null +++ b/docs/0-13-0/creating-workers/workers.mdx @@ -0,0 +1,139 @@ +--- +title: "Workers" +description: "Deploying and integrating workers into an iii project." +owner: "devrel" +type: "how-to" +--- + +## How workers expand iii + +Workers add capability to an iii system. Each one contributes functions and triggers the engine can +route to. This page covers deploying and wiring workers into a project. For the authoring surface a +worker uses to register its functions and triggers (the SDK), see each language's Worker Docs +authoring guide. + +## Connecting to the engine + +A worker connects to the engine over WebSocket. Set the engine URL via the `III_URL` environment +variable, or pass it explicitly to `register_worker`. The connection string is the only coupling +between a worker and the iii instance it joins, so the worker process can be deployed anywhere +reachable on the network. + + + + ```typescript + import { registerWorker } from "iii-sdk"; + + const worker = registerWorker(process.env.III_URL); + ``` + + + + ```python + import os + from iii import register_worker, InitOptions + + worker = register_worker( + os.environ.get("III_URL"), + InitOptions(worker_name="my-worker"), + ) + ``` + + + + ```rust + use iii_sdk::{InitOptions, register_worker}; + + let url = std::env::var("III_URL").expect("III_URL must be set"); + let worker = register_worker(&url, InitOptions::default()); + ``` + + + + +## Worker lifecycle states + +Workers transition through a small set of states after connecting: +`connecting → connected → available / busy → disconnected`. `connecting` is the WebSocket handshake. +`connected` means the Worker has joined the Engine's registry. `available` and `busy` describe +whether the Worker is currently handling invocations. `disconnected` is the terminal state when the +WebSocket closes. The Engine tracks these transitions and surfaces them to other Workers and tooling +through its discovery functions, so the rest of the system can react. + +## Handling Worker disconnects + +When a Worker's WebSocket closes, the Engine cleans up after it automatically. Its Functions and +Triggers leave the live registry, and any in-flight invocations of those Functions are cancelled. +There are two things to handle on the caller side: + +1. **Catch `invocation_stopped`.** Callers waiting on a Function whose Worker disconnects mid-flight + receive an `invocation_stopped` error rather than a timeout. Treat it like a cancellation, not a + transient failure. Retrying will fail until a Worker reconnects and re-registers the Function. +1. **Subscribe to the discovery events** if you need to react to topology changes: + - `engine::workers-available` fires immediately when a Worker connects or disconnects. + - `engine::functions-available` is eventually consistent; it fires on the next polling tick once + the function-list hash changes. + +## Inspecting the live registry + +To see what's currently connected to the Engine, use one of two surfaces depending on whether you +want a snapshot or a live subscription: + +- **Read a snapshot** by invoking one of the `engine::*::list` Functions: + - `engine::workers::list`: every connected Worker with metrics. + - `engine::functions::list`: every registered Function (filterable by `include_internal`). + - `engine::triggers::list`: every registered Trigger (filterable by `include_internal`). + - `engine::trigger-types::list`: every advertised Trigger type with its config and call schemas. +- **Subscribe to changes** by registering a Trigger against `engine::workers-available` (fires when + a Worker connects or disconnects) or `engine::functions-available` (fires when a Function is + registered or unregistered). See [Handling Worker disconnects](#handling-worker-disconnects) for + their consistency semantics. + + + These are the high-level call surfaces. For the wire-level shapes, see [Engine + protocol](/sdk-reference/engine-sdk#engine-discovery-functions). + + +## Worker manifest + +When a worker is checked into a project so iii can launch it locally, `iii.worker.yaml` at the +worker's root tells iii how to install dependencies, run the worker, and pass through configuration. + +```yaml +name: math-worker +runtime: + kind: python + package_manager: pip + entry: math_worker.py +scripts: + install: "pip install -r requirements.txt" + start: "python math_worker.py" +``` + +The manifest is metadata about _starting_ the Worker. Once the Worker is running, the WebSocket +connection to the Engine and the function registrations are what matter. A Worker started by +`iii worker add` and a Worker started by hand in a container behave identically to the Engine. + +For the full manifest field schema, see [Using iii / Workers](/using-iii/workers). + +## What a worker contributes + +Once connected, a worker exposes: + +- Functions, callable by `function_id` from anywhere in the system (see + [Using iii / Functions](/using-iii/functions)). +- Triggers it advertises, which other workers can bind their functions to (see + [Using iii / Triggers](/using-iii/triggers)). + +The SDK calls a worker uses to register these are language-specific and documented in each +language's Worker Docs authoring guide. + +## Run an ephemeral worker + +For one-shot jobs (Kubernetes Jobs, serverless containers, scheduled scripts), an SDK worker can +connect to a remote engine, register its functions, do the work, and exit. The engine cleans up the +worker's registrations on disconnect. + +Set `III_URL` to point the worker at the remote engine (see +[Connecting to the engine](#connecting-to-the-engine)), register the work the job needs to expose, +and let the process exit when the job is done. diff --git a/docs/0-13-0/creating-workers/workers.mdx.skill.md b/docs/0-13-0/creating-workers/workers.mdx.skill.md new file mode 100644 index 000000000..33cc03769 --- /dev/null +++ b/docs/0-13-0/creating-workers/workers.mdx.skill.md @@ -0,0 +1,137 @@ + + +# Workers + + +## How workers expand iii + +Workers add capability to an iii system. Each one contributes functions and triggers the engine can +route to. This page covers deploying and wiring workers into a project. For the authoring surface a +worker uses to register its functions and triggers (the SDK), see each language's Worker Docs +authoring guide. + +## Connecting to the engine + +A worker connects to the engine over WebSocket. Set the engine URL via the `III_URL` environment +variable, or pass it explicitly to `register_worker`. The connection string is the only coupling +between a worker and the iii instance it joins, so the worker process can be deployed anywhere +reachable on the network. + + + + ```typescript + import { registerWorker } from "iii-sdk"; + + const worker = registerWorker(process.env.III_URL); + ``` + + + + ```python + import os + from iii import register_worker, InitOptions + + worker = register_worker( + os.environ.get("III_URL"), + InitOptions(worker_name="my-worker"), + ) + ``` + + + + ```rust + use iii_sdk::{InitOptions, register_worker}; + + let url = std::env::var("III_URL").expect("III_URL must be set"); + let worker = register_worker(&url, InitOptions::default()); + ``` + + + + +## Worker lifecycle states + +Workers transition through a small set of states after connecting: +`connecting → connected → available / busy → disconnected`. `connecting` is the WebSocket handshake. +`connected` means the Worker has joined the Engine's registry. `available` and `busy` describe +whether the Worker is currently handling invocations. `disconnected` is the terminal state when the +WebSocket closes. The Engine tracks these transitions and surfaces them to other Workers and tooling +through its discovery functions, so the rest of the system can react. + +## Handling Worker disconnects + +When a Worker's WebSocket closes, the Engine cleans up after it automatically. Its Functions and +Triggers leave the live registry, and any in-flight invocations of those Functions are cancelled. +There are two things to handle on the caller side: + +1. **Catch `invocation_stopped`.** Callers waiting on a Function whose Worker disconnects mid-flight + receive an `invocation_stopped` error rather than a timeout. Treat it like a cancellation, not a + transient failure. Retrying will fail until a Worker reconnects and re-registers the Function. +1. **Subscribe to the discovery events** if you need to react to topology changes: + - `engine::workers-available` fires immediately when a Worker connects or disconnects. + - `engine::functions-available` is eventually consistent; it fires on the next polling tick once + the function-list hash changes. + +## Inspecting the live registry + +To see what's currently connected to the Engine, use one of two surfaces depending on whether you +want a snapshot or a live subscription: + +- **Read a snapshot** by invoking one of the `engine::*::list` Functions: + - `engine::workers::list`: every connected Worker with metrics. + - `engine::functions::list`: every registered Function (filterable by `include_internal`). + - `engine::triggers::list`: every registered Trigger (filterable by `include_internal`). + - `engine::trigger-types::list`: every advertised Trigger type with its config and call schemas. +- **Subscribe to changes** by registering a Trigger against `engine::workers-available` (fires when + a Worker connects or disconnects) or `engine::functions-available` (fires when a Function is + registered or unregistered). See [Handling Worker disconnects](#handling-worker-disconnects) for + their consistency semantics. + + + These are the high-level call surfaces. For the wire-level shapes, see [Engine + protocol](/sdk-reference/engine-sdk#engine-discovery-functions). + + +## Worker manifest + +When a worker is checked into a project so iii can launch it locally, `iii.worker.yaml` at the +worker's root tells iii how to install dependencies, run the worker, and pass through configuration. + +```yaml +name: math-worker +runtime: + kind: python + package_manager: pip + entry: math_worker.py +scripts: + install: "pip install -r requirements.txt" + start: "python math_worker.py" +``` + +The manifest is metadata about _starting_ the Worker. Once the Worker is running, the WebSocket +connection to the Engine and the function registrations are what matter. A Worker started by +`iii worker add` and a Worker started by hand in a container behave identically to the Engine. + +For the full manifest field schema, see [Using iii / Workers](/using-iii/workers). + +## What a worker contributes + +Once connected, a worker exposes: + +- Functions, callable by `function_id` from anywhere in the system (see + [Using iii / Functions](/using-iii/functions)). +- Triggers it advertises, which other workers can bind their functions to (see + [Using iii / Triggers](/using-iii/triggers)). + +The SDK calls a worker uses to register these are language-specific and documented in each +language's Worker Docs authoring guide. + +## Run an ephemeral worker + +For one-shot jobs (Kubernetes Jobs, serverless containers, scheduled scripts), an SDK worker can +connect to a remote engine, register its functions, do the work, and exit. The engine cleans up the +worker's registrations on disconnect. + +Set `III_URL` to point the worker at the remote engine (see +[Connecting to the engine](#connecting-to-the-engine)), register the work the job needs to expose, +and let the process exit when the job is done. diff --git a/docs/0-13-0/how-to/build-a-realtime-todo-app.mdx b/docs/0-13-0/how-to/build-a-realtime-todo-app.mdx new file mode 100644 index 000000000..a28fb1b40 --- /dev/null +++ b/docs/0-13-0/how-to/build-a-realtime-todo-app.mdx @@ -0,0 +1,53 @@ +--- +title: "Build a real-time todo app" +description: "End-to-end reactive todo app over a single browser-to-engine WebSocket." +owner: "devrel" +type: "how-to" +--- + +A full-stack reactive todo app over a single browser-to-engine WebSocket. Function calls and live +stream change events flow over the same connection. + +## Engine configuration + +The `config.yaml` workers needed to support a browser-driven reactive app. + +## Backend + +The worker, auth, stream wrapper, and CRUD functions that power the app server-side. + +### Worker setup + +Connect a worker, register the functions the app needs, and bind triggers. + +### Auth function (RBAC) + +An auth function runs on each new browser WebSocket connection, issues a session ID, and namespaces +the session's registered functions to prevent collisions. + +### Stream wrapper + +A typed wrapper class over `stream::get` / `stream::set` / `stream::list` / `stream::delete` for one +named stream. + +### Functions + +The CRUD functions the browser calls (create, toggle, delete, list) wired to write through the +stream wrapper. + +## Frontend + +The browser-side WebSocket connection and reactive hook that render live updates. + +### Connection + +Connect the browser to the engine over a single WebSocket using `iii-browser-sdk`. + +### Real-time hook + +A hook that subscribes to the stream and re-renders the UI on each change event the engine pushes. + +## Key concepts + +The reactive primitives this app relies on: single-WebSocket transport, per-session RBAC +namespacing, and stream-driven UI updates. diff --git a/docs/0-13-0/how-to/build-a-realtime-todo-app.mdx.skill.md b/docs/0-13-0/how-to/build-a-realtime-todo-app.mdx.skill.md new file mode 100644 index 000000000..331a1f962 --- /dev/null +++ b/docs/0-13-0/how-to/build-a-realtime-todo-app.mdx.skill.md @@ -0,0 +1,51 @@ + + +# Build a real-time todo app + + +A full-stack reactive todo app over a single browser-to-engine WebSocket. Function calls and live +stream change events flow over the same connection. + +## Engine configuration + +The `config.yaml` workers needed to support a browser-driven reactive app. + +## Backend + +The worker, auth, stream wrapper, and CRUD functions that power the app server-side. + +### Worker setup + +Connect a worker, register the functions the app needs, and bind triggers. + +### Auth function (RBAC) + +An auth function runs on each new browser WebSocket connection, issues a session ID, and namespaces +the session's registered functions to prevent collisions. + +### Stream wrapper + +A typed wrapper class over `stream::get` / `stream::set` / `stream::list` / `stream::delete` for one +named stream. + +### Functions + +The CRUD functions the browser calls (create, toggle, delete, list) wired to write through the +stream wrapper. + +## Frontend + +The browser-side WebSocket connection and reactive hook that render live updates. + +### Connection + +Connect the browser to the engine over a single WebSocket using `iii-browser-sdk`. + +### Real-time hook + +A hook that subscribes to the stream and re-renders the UI on each change event the engine pushes. + +## Key concepts + +The reactive primitives this app relies on: single-WebSocket transport, per-session RBAC +namespacing, and stream-driven UI updates. diff --git a/docs/0-13-0/index.mdx b/docs/0-13-0/index.mdx new file mode 100644 index 000000000..848be1f0d --- /dev/null +++ b/docs/0-13-0/index.mdx @@ -0,0 +1,99 @@ +--- +title: "Welcome to iii" +description: + "A next-generation software system that makes it possible to effortlessly compose, extend, and + observe every service in real-time for the first time ever." +owner: "devrel" +type: "explanation" +--- + +{/* The tabs above split the site into Docs (start here), Tutorials, How-tos, Patterns, SDK & Engine Reference, and Changelog. Read this page and the Docs tab in order first. The other tabs make more sense once you have the model. */} + +## The Problem + +Each service in a modern system arrives with its own internals, its own lifecycle, its own +integration story, and its own failure modes. Four services means six possible integration edges. +Twenty means 190. + +What we mean by edges is that every new integration brings with it cross-system interactions. For +example the addition of an agentic harness includes not just a single integration point with your +current system but the numerous downstream consumers of it. + +Every new capability quadratically compounds the coordination cost of everything already in your +stack. + +## The Solution + +iii organizes software into three primitives: **Worker**, **Trigger**, **Function**. Something hosts +work, something causes it, something does it. Every capability in every system can be built from +these three things: queues, cron, streaming, sandboxing, observability, agents, business logic, +devices, even frontend browser UIs. + +## Quadratic to Zero + +The integration problem doesn't only manifest where two systems connect. It creates downstream +friction in logging, debugging, and upgrading, and upstream friction that blocks growth initiatives, +new verticals, and partnerships. + +iii reduces all of that effort to zero. Adding two workers or two hundred is the same operation. A +new worker joins the system by opening a single WebSocket connection and becomes immediately +available to every other worker at that moment. + +## Have a Need? Add a Worker + +Need a queue? Add a worker. Need real-time streaming, scheduling, sandboxing, observability, an +agent, a CRM integration, a browser tab participating as a worker? Add a worker. Some workers are +deterministic code. Some are stochastic agents. + +`iii worker add` is the npm moment for systems. What installs is not a library. It is a complete +running service. A queue worker. A sandbox worker. A classifier. One command, complete capability, +immediately available to every other worker in the system. + +## Same Contract, Both Sides + +Application teams register functions and declare triggers, focused entirely on business logic. +Platform teams publish workers, focused entirely on the capabilities they provide. Both sides +fulfill the same contract. No bespoke SDKs, no internal client libraries, no per-service API +contracts. The work that used to live between teams disappears. + +## Any Language, Any Runtime + +A worker in Docker, on Kubernetes, on the edge, in a browser tab, on a Raspberry Pi, or inside a +hardware-isolated microVM is the same kind of worker. Moving a workload is a redeploy, not a +rewrite. The engine handles serialization and routing. + +## Built for Agents + +Most agent harnesses solve one slice of the problem: a chat loop, a tool-calling sandbox, a chained +workflow. iii is not a harness for agents. Instead it functions better than a harness because it is +the same runtime the whole system already runs on. This makes iii inherently agentic. In iii an +agent is a worker and can (with permission) act on every part of the system, not inside a runtime +created separately for agents. + +Agents are workers. An agent's tools are functions. Its memory is state. Its orchestration is +triggers. The agent does not call out to a separate "agent runtime" to do work. The runtime is the +rest of the system. An agent that hits a task outside its current capabilities can register a worker +at runtime, expose new functions, and extend the system it operates inside. That is what makes iii +different. + +Humans and agents share one mental model. A new engineer is productive on day one because the mental +model never changes from one capability to the next. AI agents can reliably reason about an entire +system in a single context window because there is one set of primitives to learn and one +always-accurate source of truth for what exists. + +As agents do more of the work of building and operating software, the size of that primitive set is +what compounds: easier to onboard, cheaper to prompt, faster to extend, simpler to maintain. + +## Getting Started + +The best way to understand iii is to try it. Install the engine and follow the Quickstart to create +your first iii-powered project. + + + + Install the iii engine. + + + Follow the Quickstart and explore a live iii application. + + diff --git a/docs/0-13-0/index.mdx.skill.md b/docs/0-13-0/index.mdx.skill.md new file mode 100644 index 000000000..cc83bafca --- /dev/null +++ b/docs/0-13-0/index.mdx.skill.md @@ -0,0 +1,95 @@ + + +# Welcome to iii + + +{/* The tabs above split the site into Docs (start here), Tutorials, How-tos, Patterns, SDK & Engine Reference, and Changelog. Read this page and the Docs tab in order first. The other tabs make more sense once you have the model. */} + +## The Problem + +Each service in a modern system arrives with its own internals, its own lifecycle, its own +integration story, and its own failure modes. Four services means six possible integration edges. +Twenty means 190. + +What we mean by edges is that every new integration brings with it cross-system interactions. For +example the addition of an agentic harness includes not just a single integration point with your +current system but the numerous downstream consumers of it. + +Every new capability quadratically compounds the coordination cost of everything already in your +stack. + +## The Solution + +iii organizes software into three primitives: **Worker**, **Trigger**, **Function**. Something hosts +work, something causes it, something does it. Every capability in every system can be built from +these three things: queues, cron, streaming, sandboxing, observability, agents, business logic, +devices, even frontend browser UIs. + +## Quadratic to Zero + +The integration problem doesn't only manifest where two systems connect. It creates downstream +friction in logging, debugging, and upgrading, and upstream friction that blocks growth initiatives, +new verticals, and partnerships. + +iii reduces all of that effort to zero. Adding two workers or two hundred is the same operation. A +new worker joins the system by opening a single WebSocket connection and becomes immediately +available to every other worker at that moment. + +## Have a Need? Add a Worker + +Need a queue? Add a worker. Need real-time streaming, scheduling, sandboxing, observability, an +agent, a CRM integration, a browser tab participating as a worker? Add a worker. Some workers are +deterministic code. Some are stochastic agents. + +`iii worker add` is the npm moment for systems. What installs is not a library. It is a complete +running service. A queue worker. A sandbox worker. A classifier. One command, complete capability, +immediately available to every other worker in the system. + +## Same Contract, Both Sides + +Application teams register functions and declare triggers, focused entirely on business logic. +Platform teams publish workers, focused entirely on the capabilities they provide. Both sides +fulfill the same contract. No bespoke SDKs, no internal client libraries, no per-service API +contracts. The work that used to live between teams disappears. + +## Any Language, Any Runtime + +A worker in Docker, on Kubernetes, on the edge, in a browser tab, on a Raspberry Pi, or inside a +hardware-isolated microVM is the same kind of worker. Moving a workload is a redeploy, not a +rewrite. The engine handles serialization and routing. + +## Built for Agents + +Most agent harnesses solve one slice of the problem: a chat loop, a tool-calling sandbox, a chained +workflow. iii is not a harness for agents. Instead it functions better than a harness because it is +the same runtime the whole system already runs on. This makes iii inherently agentic. In iii an +agent is a worker and can (with permission) act on every part of the system, not inside a runtime +created separately for agents. + +Agents are workers. An agent's tools are functions. Its memory is state. Its orchestration is +triggers. The agent does not call out to a separate "agent runtime" to do work. The runtime is the +rest of the system. An agent that hits a task outside its current capabilities can register a worker +at runtime, expose new functions, and extend the system it operates inside. That is what makes iii +different. + +Humans and agents share one mental model. A new engineer is productive on day one because the mental +model never changes from one capability to the next. AI agents can reliably reason about an entire +system in a single context window because there is one set of primitives to learn and one +always-accurate source of truth for what exists. + +As agents do more of the work of building and operating software, the size of that primitive set is +what compounds: easier to onboard, cheaper to prompt, faster to extend, simpler to maintain. + +## Getting Started + +The best way to understand iii is to try it. Install the engine and follow the Quickstart to create +your first iii-powered project. + + + + Install the iii engine. + + + Follow the Quickstart and explore a live iii application. + + diff --git a/docs/0-13-0/install.mdx b/docs/0-13-0/install.mdx new file mode 100644 index 000000000..9b040dc6a --- /dev/null +++ b/docs/0-13-0/install.mdx @@ -0,0 +1,46 @@ +--- +title: "Install" +description: "Install the iii engine and set up your development environment." +owner: "devrel" +type: "how-to" +--- + +{/* TODO (community feedback): Add a direct binary-download path alongside the curl|sh installer so users who do not want to verify a remote script with a bot, or are on Windows, have an alternative. Link to the GitHub Releases page for the engine and include PATH setup instructions per OS (especially Windows). */} + +## 1. Install iii + +Install the iii engine: + +```bash +curl -fsSL https://install.iii.dev/iii/main/install.sh | sh +``` + +## 2. Verify installation + +Check that iii has installed correctly with the following command. It should return a version +number. + +```bash +iii --version +``` + + + The engine and SDK packages can have different patch versions within the same minor line. Keep the + engine and SDKs on the same minor version, for example `0.11.x`, unless a release note says + otherwise. + + +{/* TODO: re-enable the "## 3. Install the VS Code Extension (Optional)" section once the iii-lsp extension is more thoroughly tested across VS Code, Cursor, Windsurf, and VSCodium. The Frame demo also needs `/images/lsp.mp4` to be captured and committed before the section is re-added. ## 3. Install the VS Code Extension (Optional) The iii Language Server extension gives your editor live awareness of your iii project: - **Completions**: function IDs, trigger types, payload properties, and known values like stream names and API paths. - **Hover documentation**: request and response schemas for any registered function. - **Diagnostics**: validates function IDs, required payload fields, trigger configs, cron expressions, and HTTP methods as you type. Works with TypeScript, Python, and Rust files. Requires a running iii engine.