Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed (older, pre-1.4 notes)
- Removed broken references to non-existent `security-threat-analyzer.yaml` template and `SECURITY_TEMPLATES.md` from REFERENCE.md.

## [1.6.1] - 2026-08-31

### Security
- **Declared behavioural contracts now fail closed** — if a resolved task declares a contract but the `behavioural-contracts` enforcement dependency is unavailable, execution stops before the affected model call with `CONTRACTS_UNAVAILABLE` instead of logging a warning and continuing without the promised constraint. Direct dependencies and locally delegated tasks are preflighted before their chain starts. (#103)
- **Sandbox domain rules cover ports and MCP endpoints** — `http.allow_domains` entries may pin `host:port` while bare hosts retain backwards-compatible any-port semantics. Statically configured MCP endpoints are preflighted across the selected task and direct dependencies before discovery or model execution. Malformed allowlist entries now fail validation. (#104)

### Breaking
- Existing specs that combine MCP tools with `sandbox.http.allow_domains` must add each MCP endpoint host (or `host:port`) to the effective allowlist. Runtimes now enforce the declared network boundary for MCP instead of limiting it to native HTTP tools.
- Specs that declare `behavioural_contract` must install `open-agent-spec[contracts]`; execution no longer continues without enforcement.

## [1.6.0] - 2026-07-28

### Added
Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -375,6 +375,8 @@ See [`examples/sandboxed-agent/`](examples/sandboxed-agent/).

Declare what the model output must contain. The `behavioural-contracts` library enforces the contract after parsing, before the result is returned.

Declared contracts fail closed with `CONTRACTS_UNAVAILABLE` before the affected task invokes a model when enforcement is unavailable.

```yaml
behavioural_contract:
version: "1.0"
Expand Down
2 changes: 1 addition & 1 deletion Website/app/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -252,7 +252,7 @@ tasks:

<div className="rounded-lg border border-stone-300/60 bg-stone-50/80 p-3">
<div className="mb-1 text-xs font-semibold text-stone-900">Behavioural contracts (optional)</div>
<p className="text-xs text-stone-600">Attach output contracts to tasks with the <code className="rounded bg-stone-200 px-1">behavioural-contracts</code> library. Validate required fields, confidence scores, and custom rules, after parsing, before returning. Degrades gracefully when not installed.</p>
<p className="text-xs text-stone-600">Attach output contracts to tasks with the <code className="rounded bg-stone-200 px-1">behavioural-contracts</code> library. Validate required fields, confidence scores, and custom rules after parsing and before returning. Declared contracts fail closed when enforcement is unavailable.</p>
</div>

<div className="rounded-lg border border-stone-300/60 bg-stone-50/80 p-3">
Expand Down
13 changes: 8 additions & 5 deletions docs/REFERENCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -455,7 +455,7 @@ Notes:

Behavioural contracts let you declare constraints on a task's output — required fields, policy rules, behavioural flags — and have them enforced automatically at run time by the [`behavioural-contracts`](https://pypi.org/project/behavioural-contracts/) library.

Contracts are **entirely optional**. Specs without them run exactly as before. When the library is not installed, OA logs a hard warning and continues.
Contracts are **entirely optional**. Specs without them run exactly as before. When a resolved task declares a contract but the enforcement library is not installed, OA fails closed with `CONTRACTS_UNAVAILABLE` before the affected task makes a model call. Locally delegated tasks are recursively preflighted before their containing chain starts; remote delegated tasks are checked immediately after fetch.

### Install

Expand Down Expand Up @@ -528,12 +528,12 @@ summarize → [parse] → [contract check] → return result

A contract violation on a dependency stops the chain immediately and raises `CONTRACT_VIOLATION` before the dependent task ever runs.

### Skipped cases (with warning)
### Skipped and unavailable cases

| Condition | Behaviour |
|---|---|
| `response_format: text` | Validation skipped — field checks are meaningless on raw strings |
| `behavioural-contracts` not installed | Hard warning logged; execution continues |
| `behavioural-contracts` not installed | Execution fails before the affected task's model call with `CONTRACTS_UNAVAILABLE` |
| Output is not a dict (JSON parse failed) | Validation skipped with warning |

### Error on violation
Expand Down Expand Up @@ -566,7 +566,8 @@ sandbox:
# deny: [file.write] # denylist alternative (use one or the other)
http:
allow_domains:
- api.example.com # exact match or any subdomain
- api.example.com # exact match or any subdomain, any port
- localhost:3000 # optional port pinning
file:
allow_paths:
- ./data/ # resolved to absolute paths at check time
Expand Down Expand Up @@ -597,11 +598,13 @@ All sandbox violations raise `OARunError` immediately with one of three structur
| Code | Trigger |
|------|---------|
| `SANDBOX_TOOL_VIOLATION` | Tool name not in `allow` list, or in `deny` list |
| `SANDBOX_DOMAIN_VIOLATION` | HTTP host not in `allow_domains` (for `http.get` / `http.post`) |
| `SANDBOX_DOMAIN_VIOLATION` | HTTP or MCP destination not in `allow_domains`; `host:port` rules require that port |
| `SANDBOX_PATH_VIOLATION` | File path outside `allow_paths` (for `file.read` / `file.write`) |

Path traversal (`../../`) is caught automatically — paths are resolved to absolute before comparison.

MCP endpoints are static spec configuration and are checked against the effective `http.allow_domains` policy before tool discovery or model execution. A bare hostname preserves the original any-port behaviour; use `host:port` when the agent must reach only one service on that host.

### Input immutability

Every task receives a **deep copy** of its input. Chain outputs merged into downstream inputs never mutate the caller's original dict. This is enforced at three levels:
Expand Down
4 changes: 2 additions & 2 deletions npm/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion npm/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@prime-vector/open-agent-spec",
"version": "1.6.0",
"version": "1.6.1",
"description": "Run Open Agent Spec YAML files from Node.js — no Python required.",
"keywords": [
"ai",
Expand Down
60 changes: 58 additions & 2 deletions npm/src/loader.ts
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,32 @@ const VERSION_PATTERN = /^(1\.(0\.[4-9]|[1-9]\.[0-9]+)|[2-9]\.[0-9]+\.[0-9]+)$/;
// Spec features this runtime does not implement. Per the conformance honesty
// rule, specs declaring them are REFUSED rather than silently degraded —
// particularly important for sandbox, which is a security feature.
const UNSUPPORTED_ROOT_KEYS = ["tools", "sandbox", "behavioural_contract"] as const;
const UNSUPPORTED_TASK_KEYS = ["tools", "sandbox", "behavioural_contract"] as const;
const UNSUPPORTED_ROOT_KEYS = ["tools", "sandbox"] as const;
const UNSUPPORTED_TASK_KEYS = ["tools", "sandbox"] as const;
const ALLOW_DOMAIN_PATTERN = /^(?![A-Za-z][A-Za-z0-9+.-]*:\/\/)(?:\[[^\]]+\]|[^:/\s]+)(?::[0-9]{1,5})?$/;

function validateAllowDomains(sandbox: unknown, source: string): void {
if (!sandbox || typeof sandbox !== "object" || Array.isArray(sandbox)) return;
const http = (sandbox as Record<string, unknown>)["http"];
if (!http || typeof http !== "object" || Array.isArray(http)) return;
const rules = (http as Record<string, unknown>)["allow_domains"];
if (rules === undefined) return;
if (!Array.isArray(rules)) {
fail(source, "sandbox.http.allow_domains must be an array");
}
for (const [index, rule] of rules.entries()) {
if (typeof rule !== "string" || !ALLOW_DOMAIN_PATTERN.test(rule)) {
fail(source, `sandbox.http.allow_domains[${index}] must be a host or host:port`);
}
const portMatch = rule.match(/:([0-9]{1,5})$/);
if (portMatch) {
const port = Number(portMatch[1]);
if (port < 1 || port > 65535) {
fail(source, `sandbox.http.allow_domains[${index}] port must be between 1 and 65535`);
}
}
}
}

export function loadSpecFromFile(specPath: string): OASpec {
let raw: string;
Expand Down Expand Up @@ -135,6 +159,38 @@ function validateSpec(data: Record<string, unknown>, source: string): void {
fail(source, "'tasks' must contain at least one task");
}

validateAllowDomains(data["sandbox"], source);
for (const [taskName, taskDef] of Object.entries(taskMap)) {
if (taskDef && typeof taskDef === "object") {
validateAllowDomains(
(taskDef as Record<string, unknown>)["sandbox"],
`${source}: task '${taskName}'`,
);
}
}

if ("behavioural_contract" in data) {
throw new OAError(
`${source}: spec declares 'behavioural_contract:' but this runtime cannot enforce contracts.`,
"CONTRACTS_UNAVAILABLE",
"contract",
);
}
for (const [taskName, taskDef] of Object.entries(taskMap)) {
if (
taskDef &&
typeof taskDef === "object" &&
"behavioural_contract" in (taskDef as Record<string, unknown>)
) {
throw new OAError(
`${source}: task '${taskName}' declares 'behavioural_contract:' but this runtime cannot enforce contracts.`,
"CONTRACTS_UNAVAILABLE",
"contract",
taskName,
);
}
}

// ── Unsupported feature guard (conformance honesty rule) ───────────────
for (const key of UNSUPPORTED_ROOT_KEYS) {
if (key in data) {
Expand Down
56 changes: 56 additions & 0 deletions npm/tests/contracts.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
import { describe, expect, test } from "@jest/globals";
import { parseSpec } from "../src/loader.js";

const base = `
open_agent_spec: "1.6.0"
agent:
name: contract-test
description: test
intelligence:
type: llm
engine: openai
model: gpt-4o
tasks:
run:
description: run
output: {type: object}
prompts: {system: run, user: run}
`;

describe("contract capability honesty", () => {
test("a root contract fails closed with CONTRACTS_UNAVAILABLE", () => {
const spec = `${base}\nbehavioural_contract:\n version: "1.0"\n`;
expect(() => parseSpec(spec)).toThrow(
expect.objectContaining({
code: "CONTRACTS_UNAVAILABLE",
stage: "contract",
}),
);
});

test("a task contract fails closed and identifies the task", () => {
const spec = base.replace(
" output: {type: object}",
' behavioural_contract: {version: "1.0"}\n output: {type: object}',
);
expect(() => parseSpec(spec)).toThrow(
expect.objectContaining({
code: "CONTRACTS_UNAVAILABLE",
stage: "contract",
task: "run",
}),
);
});
});

describe("sandbox declaration validation", () => {
test("a URL-shaped allowlist entry is a spec-load error before unsupported-feature refusal", () => {
const spec = `${base}\nsandbox:\n http:\n allow_domains: [https://api.example.com]\n`;
expect(() => parseSpec(spec)).toThrow(
expect.objectContaining({
code: "SPEC_LOAD_ERROR",
stage: "load",
}),
);
});
});
Loading
Loading