Skip to content

Commit a85b0ac

Browse files
committed
docs: update core executor pipeline documentation and add RFC for declarative step metadata.
Signed-off-by: tercel <tercel.yi@gmail.com>
1 parent 580f07c commit a85b0ac

File tree

4 files changed

+831
-16
lines changed

4 files changed

+831
-16
lines changed

CHANGELOG.md

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,32 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
---
99

10+
## [0.17.0] - 2026-04-04
11+
12+
### Added
13+
14+
#### Pipeline v2 — Declarative Step Metadata
15+
- **4 new BaseStep fields**`match_modules` (glob patterns for selective step execution), `ignore_errors` (fault-tolerant continuation on step failure), `pure` (no side effects, safe for dry-run/validate mode), `timeout_ms` (per-step timeout in milliseconds). Implemented across all three SDKs (Python, TypeScript, Rust).
16+
- **`PipelineContext.dry_run`** — When `true`, PipelineEngine skips steps with `pure=false`, enabling `validate()` to run user-defined pure steps automatically.
17+
- **`PipelineContext.version_hint`** — Passed through to module_lookup for version negotiation.
18+
- **`PipelineContext.executed_middlewares`** — Tracks which middleware ran for on_error recovery chain.
19+
- **`StepTrace.skip_reason`** — Records why a step was skipped: `"no_match"`, `"dry_run"`, or `"error_ignored"`.
20+
- **YAML pipeline configuration** (Python) — `pipeline` section in `apcore.yaml` with `remove`, `configure`, and `steps` directives. Step resolution via `type` (registry) and `handler` (import path).
21+
22+
### Changed
23+
24+
#### Pipeline Step Rename
25+
- **`safety_check``call_chain_guard`** — Renamed across all SDKs to accurately describe the step's purpose (call chain depth/cycle/repeat checks, not transport-level rate limiting).
26+
- **TypeScript `builtin.` prefix removed** — All built-in step names in the TypeScript SDK dropped the `builtin.` prefix for cross-SDK consistency (e.g., `builtin.context_creation``context_creation`).
27+
28+
#### Pipeline Step Order
29+
- **Steps 6 and 7 swapped**`middleware_before` now executes before `input_validation` (was the reverse). Middleware transforms are now validated by the subsequent input_validation step, aligning with the Kubernetes Mutating → Validating admission order.
30+
31+
### Fixed
32+
- **Middleware transforms now validated** — Previously, middleware modifications to inputs were either never re-validated (production code) or silently discarded (pipeline abstraction). The step order swap ensures transformed inputs pass schema validation.
33+
34+
---
35+
1036
## [0.16.0] - 2026-04-05
1137

1238
### Added

docs/api/executor-api.md

Lines changed: 16 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -613,7 +613,7 @@ executor.call(module_id, inputs, context)
613613
├─ 1. Create/validate Context
614614
│ └─ If context is None, auto-create
615615
616-
├─ 2. Call chain safety checks
616+
├─ 2. Call chain guard
617617
│ ├─ 2a. Depth check: len(call_chain) >= 32 → throw CallDepthExceededError
618618
│ ├─ 2b. Cycle detection: module_id in call_chain → throw CircularCallError
619619
│ └─ 2c. Frequency detection: call_chain.count(module_id) >= max_repeat → throw CallFrequencyExceededError
@@ -633,13 +633,13 @@ executor.call(module_id, inputs, context)
633633
│ └─ If timeout, throw ApprovalTimeoutError
634634
│ └─ If pending, throw ApprovalPendingError (Phase B)
635635
636-
├─ 6. Input validation
637-
│ └─ Validate against input_schema
638-
│ └─ If failed, throw ValidationError
639-
640-
├─ 7. Middleware before
636+
├─ 6. Middleware before
641637
│ └─ middleware.before(module_id, inputs, context)
642638
639+
├─ 7. Input validation
640+
│ └─ Validate against input_schema (includes middleware transforms)
641+
│ └─ If failed, throw ValidationError
642+
643643
├─ 8. Execute module
644644
│ └─ module.execute(inputs, context)
645645
│ └─ If failed, call middleware on_error
@@ -653,6 +653,9 @@ executor.call(module_id, inputs, context)
653653
└─ 11. Return result
654654
```
655655

656+
!!! info "BaseStep Declarative Fields"
657+
Each step in the pipeline now supports four declarative metadata fields: `match_modules` (glob patterns for selective execution), `ignore_errors` (fault-tolerant continuation), `pure` (dry-run safety marker), and `timeout_ms` (per-step timeout). See [Core Executor](../features/core-executor.md) for details.
658+
656659
### 6.2 Automatic Context Handling
657660

658661
When modules internally call other modules, Executor automatically handles Context:
@@ -682,7 +685,7 @@ result = context.executor.call(
682685
683686
┌──────────┐ depth/cycle/freq ┌──────────────────────────┐
684687
│call_chain│───────────────────▶│ error: DEPTH_EXCEEDED │
685-
│ guard │ │ / CIRCULAR_CALL │
688+
│ guard │ (was safety_check) │ / CIRCULAR_CALL │
686689
└────┬─────┘ │ / FREQUENCY_EXCEEDED│
687690
│ check passed └──────────────────────────┘
688691
@@ -703,18 +706,18 @@ result = context.executor.call(
703706
│ └──────────────────────────┘
704707
│ approved (or skipped)
705708
709+
┌──────────┐
710+
│ before │──── middleware error ──▶ on_error chain
711+
│middleware│
712+
└────┬─────┘
713+
│ transforms applied
714+
706715
┌──────────┐ validation failed ┌──────────────────────┐
707716
│ validate │────────────────────▶│ error: VALIDATION │
708717
│ input │ └──────────────────────┘
709718
└────┬─────┘
710719
│ validation passed
711720
712-
┌──────────┐
713-
│ before │──── middleware error ──▶ on_error chain
714-
│middleware│
715-
└────┬─────┘
716-
717-
718721
┌──────────┐ execution error ┌──────────────────────┐
719722
│ execute │────────────────────▶│ on_error middleware │
720723
│ module │ └──────────────────────┘

docs/features/core-executor.md

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ The executor processes every module call through the following pipeline:
2222

2323
1. **Context Creation** -- A `Context` object is constructed carrying the caller identity, call metadata, and any propagated state from parent calls. This context flows through every subsequent step.
2424

25-
2. **Safety Checks** -- Three safety mechanisms are evaluated before proceeding:
25+
2. **Call Chain Guard** -- Three safety mechanisms are evaluated before proceeding:
2626
- *Call depth check*: Rejects calls that exceed the configured maximum nesting depth, preventing unbounded recursion.
2727
- *Circular call detection*: Inspects the call chain recorded in the context to detect and reject circular module invocations.
2828
- *Frequency throttling*: Tracks call frequency per module and rejects calls that exceed the configured rate, protecting against tight-loop abuse.
@@ -33,9 +33,9 @@ The executor processes every module call through the following pipeline:
3333

3434
5. **Approval Gate** -- If an `ApprovalHandler` is configured and the module declares `requires_approval=true`, the handler is invoked to obtain approval before proceeding. The handler may block for human input or return immediately. Rejected, timed-out, or still-pending approvals raise `ApprovalDeniedError`, `ApprovalTimeoutError`, or `ApprovalPendingError` respectively. Skipped entirely when no handler is configured or the module does not require approval. See [Approval System](./approval-system.md).
3535

36-
6. **Input Validation with Pydantic + Sensitive Field Redaction** -- The call's input payload is validated against the module's input schema (a dynamically generated Pydantic model). Fields annotated with `x-sensitive` are redacted from logs and error messages using the `redact_sensitive` utility.
36+
6. **Middleware Before Chain** -- All registered "before" middleware functions are executed in order. Each middleware receives the context and input, and may modify or enrich them before validation runs.
3737

38-
7. **Middleware Before Chain** -- All registered "before" middleware functions are executed in order. Each middleware receives the context and validated input, and may modify or enrich them before the module runs.
38+
7. **Input Validation with Pydantic + Sensitive Field Redaction** -- The call's input payload (including any modifications from middleware) is validated against the module's input schema (a dynamically generated Pydantic model). Fields annotated with `x-sensitive` are redacted from logs and error messages using the `redact_sensitive` utility.
3939

4040
8. **Module Execution with Timeout (Dual-Timeout Model)** -- The module's handler is invoked with dual-timeout enforcement: both a per-module timeout (`resources.timeout`, default 30s) and a global deadline (`executor.global_timeout`, default 60s). The shorter of the two is applied, preventing nested call chains from exceeding the global budget. The global deadline is set on the root call and propagated to child contexts via `Context._global_deadline`.
4141

@@ -47,6 +47,18 @@ The executor processes every module call through the following pipeline:
4747

4848
11. **Result Return** -- The final validated output (or error) is packaged into a structured result and returned to the caller.
4949

50+
!!! info "Step Metadata"
51+
Each pipeline step declares four metadata fields:
52+
53+
| Field | Type | Default | Purpose |
54+
|-------|------|---------|---------|
55+
| `match_modules` | glob patterns or null | `null` (all) | Only run this step for matching module IDs |
56+
| `ignore_errors` | bool | `false` | If true, step failure logs warning and continues |
57+
| `pure` | bool | `false` | If true, safe to run during `validate()` dry-run mode |
58+
| `timeout_ms` | int | `0` | Per-step timeout in milliseconds (0 = no limit) |
59+
60+
These fields enable targeted step application, fault-tolerant pipelines, and dry-run validation without code changes.
61+
5062
### Key Classes
5163

5264
- **Executor** -- The main engine class that implements the execution pipeline. Manages middleware registration, timeout configuration, and the execution loop.

0 commit comments

Comments
 (0)