Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions LIMITATIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,10 @@ Current `0.1.0-dev` limitations:
- OTel span events, Logs, and basic metrics are implemented against caller-owned
providers; collector/backend interoperability is not yet exercised.
- No metrics projector or AGT adapter is implemented.
- TRACE finalization is software-only, requires explicitly complete evidence and
trusted caller configuration, and cannot yet emit a tool transcript.
- TRACE finalization is software-only and requires explicitly complete evidence
plus trusted caller configuration.
- Action telemetry records resolved attempts only; it does not expose in-flight
lifecycle transitions.
- Evidence memory mode is not durable. Callback mode defines acknowledgement and
retry behavior but the adopter owns storage, idempotency, and recovery.
- Propagation currently supports mutable string mappings; framework-specific HTTP,
Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ The application keeps its collector, backend, policy engine, workflow framework,
| Approval lifecycle | Requested through terminal decision and execution outcome, bound to an action digest |
| Usage | Per-call/run token and cost facts with explicit cost provenance |
| Data flow | Classified source-to-destination metadata without payload capture |
| Action execution | Resolved tool, MCP, A2A, file, HTTP, and database attempts |
| Evidence lifecycle | Run checkpoints, completeness, and optional TRACE finalization status |

## Current contents
Expand Down Expand Up @@ -94,6 +95,7 @@ python examples/manual_governance.py

- [Architecture](docs/architecture.md)
- [OpenTelemetry projection](docs/otel-projection.md)
- [Action execution events](docs/action-events.md)
- [Evidence chain profile](docs/evidence-chain.md)
- [TRACE finalization](docs/trace-finalization.md)
- [Privacy](PRIVACY.md)
Expand Down
2 changes: 1 addition & 1 deletion ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

- Additional metric views and collector interoperability fixtures.
- Additional W3C propagation carrier adapters beyond mutable string mappings.
- Dedicated tool/action events and TRACE tool-transcript mapping.
- Action lifecycle expansion if adopters need in-flight attempt telemetry.
- AGT, OPA, Cedar, and generic approval adapters.
- TypeScript SDK and mixed-language conformance.

Expand Down
16 changes: 16 additions & 0 deletions conformance/fixtures/invalid/action-without-digest.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"spec_version": "0.1.0-dev",
"event_id": "018f0f7d-7a13-7cc2-8000-000000000096",
"event_type": "action.executed",
"time_unix_nano": 1787079650000000000,
"run_id": "run-governed-sdlc-001",
"agent_id": "spiffe://example.test/agent/coder",
"producer": {"name": "invalid-action-adapter", "version": "0.1.0"},
"action_id": "tool-call-missing-binding",
"action_kind": "tool",
"action_name": "filesystem.write",
"operation": "write",
"outcome": "success",
"duration_ns": 1000
}

28 changes: 28 additions & 0 deletions conformance/fixtures/valid/action.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{
"spec_version": "0.1.0-dev",
"event_id": "018f0f7d-7a13-7cc2-8000-000000000006",
"event_type": "action.executed",
"time_unix_nano": 1787079650000000000,
"run_id": "run-governed-sdlc-001",
"workflow_id": "workflow-build-feature",
"agent_id": "spiffe://example.test/agent/coder",
"producer": {"name": "example-action-adapter", "version": "0.1.0"},
"action_id": "tool-call-001",
"action_kind": "mcp",
"action_name": "github.create_pull_request",
"operation": "create",
"outcome": "success",
"duration_ns": 42000000,
"action_digest": {
"algorithm": "sha256",
"value": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"
},
"result_digest": {
"algorithm": "sha256",
"value": "eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee"
},
"target": {"kind": "git.repository", "id": "example/repository"},
"policy_event_id": "018f0f7d-7a13-7cc2-8000-000000000001",
"approval_id": "approval-001"
}

1 change: 1 addition & 0 deletions conformance/runner/validate.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
FIXTURE_DIR = ROOT / "conformance" / "fixtures"

EVENT_SCHEMAS = {
"action.executed": "action.schema.json",
"policy.decision": "policy-decision.schema.json",
"usage.recorded": "usage.schema.json",
"data_flow.observed": "data-flow.schema.json",
Expand Down
19 changes: 19 additions & 0 deletions docs/action-events.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Action execution events

`action.executed` records one resolved action attempt. It covers tools, MCP,
A2A, file, HTTP, database, and explicitly identified other actions.

The event requires an `action_digest`: a digest of the canonical authorization
subject used by the producer. An approval event can bind to the same digest, and
the action may additionally name its `approval_id` and governing
`policy_event_id`. The telemetry contract validates these fields but does not
prove that a producer computed or linked them honestly.

`outcome` distinguishes success, error, denial, cancellation, and timeout.
Denied attempts are still resolved attempts and therefore count in a TRACE tool
transcript. The event intentionally carries no arguments, results, source code,
credentials, or authorization material; optional digests identify those objects
without capturing them.

This revision does not represent started/in-flight actions. A producer emits the
event once the attempt has a terminal outcome and records its total duration.
2 changes: 1 addition & 1 deletion docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ policy / approval / usage / data hooks

### Normative contract

JSON Schema 2020-12 files in `spec/schema` define the five event families. Closed schemas reject unknown top-level fields. A shared envelope supplies event, run, workflow, agent, task, producer, trace, and span correlation.
JSON Schema 2020-12 files in `spec/schema` define the six event families. Closed schemas reject unknown top-level fields. A shared envelope supplies event, run, workflow, agent, task, producer, trace, and span correlation.

### Conformance suite

Expand Down
3 changes: 3 additions & 0 deletions docs/otel-projection.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ When a recording span is active, the SDK calls `add_event` with:
| `policy_decision` | `agentrust.data_flow.policy_decision` |
| `capture_profile` | `agentrust.evidence.capture_profile` |
| `completeness` | `agentrust.evidence.completeness` |
| `action_kind` | `agentrust.action.kind` |
| `outcome` | `agentrust.action.outcome` |

Nested normalized objects are projected only through explicit mappings. They are never serialized wholesale into span attributes. Raw content fields are prohibited before projection.

Expand All @@ -53,6 +55,7 @@ caller-owned Logger and never installs a provider or exporter.

- `agentrust.policy.decisions` and `agentrust.policy.evaluation.duration`;
- `agentrust.approval.events`;
- `agentrust.action.executions` and `agentrust.action.duration`;
- `agentrust.data_flow.events`;
- `agentrust.usage.tokens` and `agentrust.usage.cost`.

Expand Down
6 changes: 4 additions & 2 deletions docs/trace-finalization.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,10 @@ Finalization fails when evidence is open, incomplete, empty, inconsistently boun
to policy, missing classified data flows, or contains an unranked classification.
It emits `runtime.platform: software-only`; it cannot manufacture attestation.

No `tool_transcript` is emitted yet because the telemetry contract has no
dedicated tool-call event. Governance-event count is not tool-call count.
When `action.executed` events are present, `tool_transcript.hash` covers their
normalized bytes and evidence sequence in acceptance order, and `call_count`
equals the number of those action events. Other governance events are excluded.
When no action evidence is present, the optional transcript remains absent.

The signed record remains subject to TRACE's documented trust-anchor, freshness,
revocation, transparency, and software-only limitations.
40 changes: 40 additions & 0 deletions spec/schema/action.schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://agentrust.io/telemetry/v0.1/schema/action.schema.json",
"allOf": [
{"$ref": "envelope.schema.json"},
{
"type": "object",
"required": [
"agent_id", "event_type", "action_id", "action_kind", "action_name",
"operation", "outcome", "duration_ns", "action_digest"
],
"properties": {
"event_type": {"const": "action.executed"},
"action_id": {"$ref": "common.schema.json#/$defs/nonempty"},
"action_kind": {"enum": ["tool", "mcp", "a2a", "file", "http", "database", "other"]},
"action_name": {"$ref": "common.schema.json#/$defs/nonempty"},
"operation": {"$ref": "common.schema.json#/$defs/nonempty"},
"outcome": {"enum": ["success", "error", "denied", "cancelled", "timeout"]},
"duration_ns": {"type": "integer", "minimum": 0},
"action_digest": {"$ref": "common.schema.json#/$defs/digest"},
"result_digest": {"$ref": "common.schema.json#/$defs/digest"},
"target": {
"type": "object",
"required": ["kind"],
"properties": {
"kind": {"$ref": "common.schema.json#/$defs/nonempty"},
"id": {"$ref": "common.schema.json#/$defs/nonempty"}
},
"additionalProperties": false
},
"policy_event_id": {"$ref": "common.schema.json#/$defs/event_id"},
"approval_id": {"$ref": "common.schema.json#/$defs/nonempty"},
"remote_agent_id": {"$ref": "common.schema.json#/$defs/nonempty"},
"error_type": {"$ref": "common.schema.json#/$defs/nonempty"}
}
}
],
"unevaluatedProperties": false
}

14 changes: 14 additions & 0 deletions src/agentrust_telemetry/otel.py
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,12 @@ def __init__(
self._approval_count = meter.create_counter(
"agentrust.approval.events", unit="{event}", description="Approval lifecycle events"
)
self._action_count = meter.create_counter(
"agentrust.action.executions", unit="{execution}", description="Resolved action attempts"
)
self._action_duration = meter.create_histogram(
"agentrust.action.duration", unit="s", description="Resolved action duration"
)
self._data_flow_count = meter.create_counter(
"agentrust.data_flow.events", unit="{event}", description="Classified data-flow events"
)
Expand Down Expand Up @@ -80,6 +86,14 @@ def emit(self, event: dict[str, Any]) -> bool:
},
)
return True
if event_type == "action.executed":
attributes = {
"agentrust.action.kind": event["action_kind"],
"agentrust.action.outcome": event["outcome"],
}
self._action_count.add(1, attributes)
self._action_duration.record(event["duration_ns"] / 1_000_000_000, attributes)
return True
if event_type == "data_flow.observed":
value = event["classification"]["value"]
classification = value if value in self._classification_values else "_other"
Expand Down
2 changes: 2 additions & 0 deletions src/agentrust_telemetry/projection.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@
"policy_decision": "agentrust.data_flow.policy_decision",
"capture_profile": "agentrust.evidence.capture_profile",
"completeness": "agentrust.evidence.completeness",
"action_kind": "agentrust.action.kind",
"outcome": "agentrust.action.outcome",
}


Expand Down
40 changes: 40 additions & 0 deletions src/agentrust_telemetry/schemas/action.schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://agentrust.io/telemetry/v0.1/schema/action.schema.json",
"allOf": [
{"$ref": "envelope.schema.json"},
{
"type": "object",
"required": [
"agent_id", "event_type", "action_id", "action_kind", "action_name",
"operation", "outcome", "duration_ns", "action_digest"
],
"properties": {
"event_type": {"const": "action.executed"},
"action_id": {"$ref": "common.schema.json#/$defs/nonempty"},
"action_kind": {"enum": ["tool", "mcp", "a2a", "file", "http", "database", "other"]},
"action_name": {"$ref": "common.schema.json#/$defs/nonempty"},
"operation": {"$ref": "common.schema.json#/$defs/nonempty"},
"outcome": {"enum": ["success", "error", "denied", "cancelled", "timeout"]},
"duration_ns": {"type": "integer", "minimum": 0},
"action_digest": {"$ref": "common.schema.json#/$defs/digest"},
"result_digest": {"$ref": "common.schema.json#/$defs/digest"},
"target": {
"type": "object",
"required": ["kind"],
"properties": {
"kind": {"$ref": "common.schema.json#/$defs/nonempty"},
"id": {"$ref": "common.schema.json#/$defs/nonempty"}
},
"additionalProperties": false
},
"policy_event_id": {"$ref": "common.schema.json#/$defs/event_id"},
"approval_id": {"$ref": "common.schema.json#/$defs/nonempty"},
"remote_agent_id": {"$ref": "common.schema.json#/$defs/nonempty"},
"error_type": {"$ref": "common.schema.json#/$defs/nonempty"}
}
}
],
"unevaluatedProperties": false
}

25 changes: 25 additions & 0 deletions src/agentrust_telemetry/trace_adapter.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

from __future__ import annotations

import hashlib
import json
from dataclasses import dataclass
from typing import Any, Literal

Expand Down Expand Up @@ -43,6 +45,7 @@ def finalize_trace(

bundle_hash, enforcement_mode = _policy_binding(events)
data_class = _highest_data_class(events, config)
tool_transcript = _tool_transcript(snapshot)
record: dict[str, Any] = {
"eat_profile": trace.TRACE_PROFILE_V0_2,
"iat": max(event["time_unix_nano"] for event in events) // 1_000_000_000,
Expand All @@ -61,6 +64,7 @@ def finalize_trace(
"enforcement_mode": enforcement_mode,
},
"data_class": data_class,
**({"tool_transcript": tool_transcript} if tool_transcript else {}),
"origin": {
"kind": config.origin_kind,
"producer": config.origin_producer,
Expand Down Expand Up @@ -188,6 +192,27 @@ def _appraisal(events: list[dict[str, Any]]) -> str:
return "none"


def _tool_transcript(snapshot: EvidenceSnapshot) -> dict[str, Any] | None:
actions = [
{"sequence": entry.sequence, "event": entry.event}
for entry in snapshot.entries
if entry.event["event_type"] == "action.executed"
]
if not actions:
return None
canonical = json.dumps(
actions,
sort_keys=True,
separators=(",", ":"),
ensure_ascii=False,
allow_nan=False,
).encode("utf-8")
return {
"hash": "sha256:" + hashlib.sha256(canonical).hexdigest(),
"call_count": len(actions),
}


def _trace_package() -> Any:
try:
import agentrust_trace
Expand Down
1 change: 1 addition & 0 deletions src/agentrust_telemetry/validation.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@


EVENT_SCHEMAS = {
"action.executed": "action.schema.json",
"policy.decision": "policy-decision.schema.json",
"usage.recorded": "usage.schema.json",
"data_flow.observed": "data-flow.schema.json",
Expand Down
9 changes: 9 additions & 0 deletions tests/test_conformance.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,15 @@ def test_uppercase_trace_id_is_rejected(self):
record["trace_id"] = record["trace_id"].upper()
self.assertTrue(validate.validate_record(record))

def test_action_payload_field_is_rejected_by_schema_and_privacy_gate(self):
record = json.loads(
(ROOT / "conformance" / "fixtures" / "valid" / "action.json").read_text()
)
record["tool_arguments"] = {"path": "sensitive.py"}
errors = validate.validate_record(record)
self.assertTrue(any("unevaluated" in error.lower() for error in errors))
self.assertTrue(any("tool_arguments" in error and "privacy" in error for error in errors))


if __name__ == "__main__":
unittest.main()
16 changes: 16 additions & 0 deletions tests/test_otel_projectors.py
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,22 @@ def create_histogram(self, *args, **kwargs):
Meter(), classification_values=frozenset(f"class-{i}" for i in range(65))
)

def test_action_metrics_exclude_action_and_agent_identifiers(self):
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import InMemoryMetricReader

reader = InMemoryMetricReader()
meter = MeterProvider(metric_readers=[reader]).get_meter("agentrust-action-test")
emitter = OTelMetricEmitter(meter)
self.assertTrue(emitter.emit(fixture("action.json")))
metrics = reader.get_metrics_data().resource_metrics[0].scope_metrics[0].metrics
for metric in metrics:
for point in metric.data.data_points:
self.assertEqual(
set(point.attributes),
{"agentrust.action.kind", "agentrust.action.outcome"},
)


if __name__ == "__main__":
unittest.main()
18 changes: 18 additions & 0 deletions tests/test_trace_adapter.py
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,24 @@ def test_finalizes_signs_and_validates_official_trace_record(self):
self.assertEqual(record["appraisal"]["status"], "contraindicated")
self.assertNotIn("tool_transcript", record)

def test_action_events_create_truthful_tool_transcript(self):
action = fixture("action.json")
snapshot = self.snapshot(extra_events=(action,))
record = finalize_trace(snapshot, self.config, signing_key=self.key)
self.assertEqual(record["tool_transcript"]["call_count"], 1)
self.assertRegex(record["tool_transcript"]["hash"], r"^sha256:[0-9a-f]{64}$")

changed = fixture("action.json")
changed["outcome"] = "error"
changed["error_type"] = "remote_error"
changed_record = finalize_trace(
self.snapshot(extra_events=(changed,)), self.config, signing_key=self.key
)
self.assertNotEqual(
record["tool_transcript"]["hash"],
changed_record["tool_transcript"]["hash"],
)

def test_refuses_unsealed_or_incomplete_evidence(self):
accumulator = EvidenceAccumulator("run-governed-sdlc-001", self.validator)
accumulator.append(fixture("policy-decision.json"))
Expand Down