feat: decision architecture (invariants, packs, receipts, approvals)

BLOCKERS.md

@@ -0,0 +1,41 @@
+# Blockers
+
+Open items that need a founder action before this branch can ship.
+
+## receipt-writer image
+
+The chart `receipts` block points at
+`ghcr.io/clawdlinux/agentic-operator-core/receipt-writer`. No CI job publishes
+that image yet. Build it with `docker build --target receipt-writer .` and push
+it, or add a release job.
+
+## Demo-only image
+
+`scripts/demo-claims.sh` still builds linux binaries on the host and packages
+them with `scripts/demo/Dockerfile.demo`. The repo Dockerfile now builds
+(agentgate v0.1.4 is pinned). Switch the demo to the repo Dockerfile targets
+and delete `Dockerfile.demo` when convenient.
+
+## Known review findings
+
+An independent review of this branch found these gaps. They are documented, not fixed.
+
+- **F7. Dataset content is not bound.** `pkg/dataset/dataset.go` (`Validate`) checks identity and label fields and the original seq. It does not bind features, content hashes, captured params, timestamps or edit content. Someone who can edit an export can poison training features and `agentctl dataset verify` still passes. Fix: bind each example to its verified original decision record and rebuild features from it.
+- **F11. Writer recovery trusts local files.** `pkg/receipts/local.go` checks hashes and the head KID on restart. It does not verify signatures or keep a committed-head checkpoint. Removing the tail of the chain rewinds the sequence and the writer can fork an already issued chain. Fix: verify signatures on recovery and persist an external checkpoint of the committed head.
+- **F12. Deleted approval rows go unnoticed.** `pkg/dataset/verify.go` checks the examples it is given. Without a signed dataset manifest (count, order, digest) it cannot tell that rows were removed. Fix: sign a dataset manifest and verify completeness against it.
+
+## Known gaps (not blockers for this branch)
+
+- Runtime-adapter parity: `reconcileViaRuntime` runs no invariants, packs,
+  receipts, or approvals. Packs on that path fail closed.
+- Argo approval gates are not covered by human approvals.
+- Execution outcomes are not receipted. Only the pre-execution decision is.
+- Chart egress: the default-deny NetworkPolicy has no rule for the workload
+  MCP endpoint. On an enforcing CNI (kindnet included) set
+  `networkPolicy.additionalAllowedHosts`.
+- A Completed direct-path workload is reconciled every 30s and proposes and
+  runs a new action each time. Pre-existing behavior, unchanged here.
+- `callerIdentity` is always empty.
+- `TestControllers` needs envtest binaries (`bin/k8s` or
+  `/usr/local/kubebuilder/bin`). It fails locally without them.
+- **Denied direct-path workloads are not terminal.** A `PolicyDenied` or `Completed` direct-path workload is proposed again on any later reconcile (stale event, hourly or 30s requeue). Each proposal is fully re-decided and receipted, so no guard is skipped. But a deny caused by a temporary condition, such as an unreachable receipt writer under INV-05, can be followed by an execution seconds later. The demo hit this once. This predates the branch. Fix: make terminal phases sticky until the spec generation changes.

CHANGELOG.md

@@ -7,11 +7,52 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+Decision architecture series. The direct action path now decides in layers:
+invariants, policy packs, a threshold evaluator, an escalate-only decision
+model in shadow, and human approval. Each decision can get a signed receipt
+before the action runs. `scripts/demo-claims.sh` proves this on kind. The
+runtime-adapter path, Argo approval gates, and execution outcomes are not
+covered yet. See [decision architecture](docs/architecture/decision-architecture.md).
+
+### Fixed
+- Status-only updates no longer re-trigger Reconcile. Before, each status write re-ran the direct-path action in a hot loop.
+- The controller no longer logs MCP status, proposal, or execution values. They can carry personal data or credentials.
+
+### Fixed (review findings)
+- An incomplete data scan (budget, depth or size limit) now denies through INV-06 instead of failing open.
+- Agent-claimed inputs can only raise model risk. The model scores with and without claims and takes the higher value.
+- The direct path reserves execution with a status update before it runs an allowed action, so a stale reconcile cannot run after the spec tightens.
+- The remote receipt writer client verifies each receipt under a pinned writer key.
+- `agentctl receipts verify` and `agentctl dataset verify` need a signed manifest by default. `--allow-prefix` is the weaker mode.
+- Receipts bind the executed payload by digest (`payload_sha256`).
+- The approval stamp carries an HMAC, and a consumed approval cannot be replayed.
+- `agentctl` approval patches carry the resourceVersion and fail on conflict.
+- A configured model that fails to load is treated as unavailable. Escalate mode then requires approval.
+- Approver identities are stored as digests in receipts.
+- MCP error bodies and invalid values are no longer logged.
+
 ### Added
+- Added `scripts/demo-claims.sh`, a no-API-key claims demo on a fresh kind cluster with a mock MCP server. It runs with a pinned writer key and an approval stamp key, and proves the review fixes: wrong pin, removed manifest, approval replay, identity digests, and INV-06. Two fresh runs passed in 252s and 297s. It builds a demo-only image from host binaries until the receiptspec dependency is tagged.
+- Added P7 decision spans on direct proposals and stamped human approve, reject, and edit decisions.
+	Successful receipt appends bind spans to writer-returned sequence and entry hash.
+	Added privacy canary tests, no-op checks, write-ahead assertions, and `make trace-coverage` with an 11-path matrix.
+	Runtime adapters, execution outcomes, receipt-writer internals, and writer trace propagation remain untraced.
+	See [decision tracing](docs/TRACING.md#decision-tracing).
 - Added a working booth prepare/present flow with real provider and cost proof.
 - Added `agentctl doctor sandbox` to verify RuntimeClass and ready-node sandbox evidence before workload deployment.
+- Added optional `spec.declaredIntent` on `AgentWorkload` and observed decision inputs. The legacy action path now denies or escalates actions that send undeclared data classes or call undeclared hosts. Agent-claimed confidence and cluster health can only tighten. New packages: `pkg/dataclass`, `pkg/decision`, `pkg/decision/input`, `pkg/decision/observe`. See `docs/policy-input.md`.
+- Added invariants (`pkg/invariants`, INV-01 to INV-06; INV-06 denies a payload the data class scan could not fully read), an embedded OPA rule engine (`pkg/rules/engine`, OPA v1.19.1), and opt-in policy packs `dpdp-in@v0.1.0` and `gdpr-eu@v0.1.0` via `spec.policyPacks`. Packs only add deny or require approval. Packs are engineering controls, not legal advice. The `credential` data class is now detected and always denied (INV-01), even without `declaredIntent`. Direct action path only. See `docs/policy-packs.md`.
+- Added opt-in signed decision receipts (`pkg/receipts`, `cmd/receipt-writer`). Each direct-path decision gets an AgentGate-format Ed25519 receipt before the action runs. `RECEIPTS_REQUIRED=true` makes INV-05 deny when no receipt can be written. Added `agentctl receipts export` and `agentctl receipts verify`, and a `global.receipts` chart block. Records hold hashes and class names, not raw params or matched data. Execution outcome and the runtime-adapter path are not receipted yet. Uses the public `pkg/receiptspec` from `github.com/Clawdlinux/agentgate` v0.1.4. See `docs/receipts.md`.
+- Added human approvals on the direct action path (`pkg/approval`, `pkg/dataset`). A held action is stored in `status.pendingApproval`. Humans decide with `clawdlinux.org/approval-decision` (approve, reject, edit). The webhook stamps the approver; the controller refuses unstamped decisions. Each decision gets a layer `human` receipt and an example in `approvals.jsonl` on the receipt-writer (`POST|GET /v1/approvals`). Edited and approved actions re-run invariants and packs first. Added `spec.approvalCapture.content` (none, redacted, full; default redacted), phase `Rejected`, `agentctl edit-approve`, `agentctl approve --reason`, and `agentctl dataset export|verify`. See `docs/approvals.md`.
+- Added the decision model, phase A (`pkg/decision` features and `ApplyModel`, `pkg/decision/learned`, `pkg/decision/baseline`, `pkg/decision/eval`). Escalate-only: it can move allow to require_approval and never loosens. `spec.decisionModel.mode` (off, shadow, escalate; default shadow) and a stricter-only `thresholdMicro`. The operator loads an artifact from `DECISION_MODEL_PATH`; without one nothing is scored. Scored decisions carry a receipt `model` block with integer micro-unit probabilities. Added `agentctl decision eval`, `make decision-train` (stdlib Python, byte-identical reruns), and `global.decisionModel` in the chart. The shipped artifact is trained on 30 DRAFT synthetic scenarios and is not production-validated. Approval examples gain an optional `source` field; `synthetic` rows never verify. See `docs/architecture/decision-model.md`.
 
 ### Changed
+- Moved the threshold evaluator from `pkg/opa` to `pkg/rules/threshold`. `pkg/opa` remains as a deprecated alias package.
+- `agentctl` table output now uses tablewriter v1 (box-drawn borders). OPA v1.19.1 requires it.
+- An unknown or unenforceable `spec.policyPacks` entry fails the workload closed with condition `PolicyPackInvalid`. The validating webhook also rejects unknown packs.
+- The AgentWorkload webhooks now register handlers. Before, `SetupWebhookWithManager` registered none, so enabling `agentWorkloadEnabled` sent requests to missing paths.
+- A `PendingApproval` workload with a stored pending action waits for a human decision instead of re-proposing every hour.
+- Docs: removed claims not backed by code. Added the decision architecture ADR (`docs/architecture/decision-architecture.md`). Rego files are labelled samples; the action path is a Go threshold evaluator.
 - Changed the internal Go `CostReporter.RecordUsage` and `Provider.CallModel` interfaces to carry operation IDs for idempotency. This breaks out-of-tree implementations.
 - Added the optional `litellm.anthropicKey` chart value. Use `litellm.existingSecret` for booth and production deployments.
 

Dockerfile

@@ -14,6 +14,19 @@ RUN CGO_ENABLED=0 GOOS=linux go build -a -o manager \
     -trimpath \
     ./cmd/main.go
 
+# receipt-writer image. Build with: docker build --target receipt-writer .
+FROM builder AS receipt-writer-build
+RUN CGO_ENABLED=0 GOOS=linux go build -o receipt-writer \
+    -ldflags="-s -w" \
+    -trimpath \
+    ./cmd/receipt-writer
+
+FROM gcr.io/distroless/static:nonroot AS receipt-writer
+COPY --from=receipt-writer-build /build/receipt-writer /receipt-writer
+USER nonroot:nonroot
+ENTRYPOINT ["/receipt-writer"]
+
+# Default target: the operator manager. Keep this stage last.
 FROM gcr.io/distroless/static:nonroot
 
 LABEL maintainer="operator-maintainers@example.com"

Makefile

@@ -18,7 +18,7 @@ SETUP_ENVTEST := $(LOCALBIN)/setup-envtest
 STATICCHECK := $(LOCALBIN)/staticcheck
 CONTROLLER_GEN := $(LOCALBIN)/controller-gen
 
-.PHONY: help test validate test-unit test-go test-anf-snapshot vet-anf-snapshot verify-anf-snapshot test-python setup-envtest test-controller fmt fmt-anf-snapshot fmt-check fmt-check-anf-snapshot vet lint staticcheck-anf-snapshot build build-agentctl build-agentctl-web build-anf-snapshot install-agentctl scan-secrets clean-venv check-python-version helm-lint test-cluster test-smoke test-e2e-cluster manifests generate
+.PHONY: help test validate test-unit test-go test-anf-snapshot vet-anf-snapshot verify-anf-snapshot test-python setup-envtest test-controller fmt fmt-anf-snapshot fmt-check fmt-check-anf-snapshot vet lint staticcheck-anf-snapshot build build-agentctl build-agentctl-web build-receipt-writer build-anf-snapshot install-agentctl scan-secrets clean-venv check-python-version helm-lint test-cluster test-smoke test-e2e-cluster manifests generate decision-train
 
 .DEFAULT_GOAL := help
 
@@ -177,6 +177,13 @@ build:
 	@echo "Building Go binaries..."
 	@$(GO) build ./...
 
+decision-train: ## Seed DRAFT synthetic data and train the decision model twice; fails unless both artifacts match
+	@bash tools/decision-train/run.sh
+
+.PHONY: trace-coverage
+trace-coverage:
+	@$(GO) test ./internal/controller -run '^TestDecisionTraceCoverage$$' -count=1 -v
+
 build-agentctl:
 	@echo "Building agentctl binary..."
 	@mkdir -p bin
@@ -187,6 +194,11 @@ build-agentctl-web:
 	@mkdir -p bin
 	@$(GO) build -o bin/agentctl-web ./cmd/agentctl-web/...
 
+build-receipt-writer:
+	@echo "Building receipt-writer binary..."
+	@mkdir -p bin
+	@$(GO) build -o bin/receipt-writer ./cmd/receipt-writer
+
 build-anf-snapshot:
 	@echo "Building anf-snapshot binary..."
 	@mkdir -p bin