Improve model promotion evidence gates

skills/ai-security/model-supply-chain/SKILL.md

@@ -14,7 +14,7 @@ phase: [build, review, operate]
 frameworks: [OWASP-LLM03-2025, SLSA-v1.0, MITRE-ATLAS]
 difficulty: advanced
 time_estimate: "45-90min"
-version: "1.0.0"
+version: "1.0.1"
 author: unitoneai
 license: MIT
 allowed-tools: Read, Grep, Glob
@@ -82,6 +82,7 @@ Before beginning the assessment, gather the following. If any item is unavailabl
 | Model signing or attestation | CI/CD configs, SLSA provenance files, Sigstore artifacts | Confirms cryptographic supply chain verification |
 | Access controls on model storage | Cloud storage IAM, artifact registry permissions | Determines who can replace or modify model weights |
 | Adapter/plugin sources | LoRA configs, adapter download code | Third-party adapters inherit the same supply chain risks |
+| Promotion and deployment manifests | Model registry releases, deployment YAML, rollout logs, approval tickets | Proves evaluated artifacts are the artifacts deployed |
 
 ---
 
@@ -229,6 +230,57 @@ Glob: **/Jenkinsfile
 
 ---
 
+### Step 3a -- Model Promotion and Deployment Binding
+
+Verify that the artifact that passed evaluation, approval, and model-card review is the exact immutable artifact deployed to each runtime environment. Provenance and evaluation evidence are incomplete if production pulls `main`, `latest`, mutable bucket prefixes, or unbound adapter paths after approval.
+
+**What to look for in code and configuration:**
+
+- Evaluation reports that reference a model family, repo, or registry alias without a digest, commit SHA, provider version ID, or immutable object version.
+- Approval tickets that approve a model name but do not bind the evaluated artifact identity, model card version, evaluation run ID, and deploy manifest.
+- Deployment manifests that use `latest`, `main`, unversioned S3/GCS paths, mutable container tags, or current provider defaults.
+- Canary, shadow, staging, and production deployments that do not record which exact artifact was served in each environment.
+- LoRA, QLoRA, PEFT, or other adapter releases that bind only the adapter and omit the base model digest, tokenizer version, or merged artifact hash.
+- Rollback plans that point to an alias or bucket prefix without verifying checksum, signature, evaluation status, and known-vulnerability or backdoor-test status.
+
+**Detection methods using allowed tools:**
+
+```
+# Find model promotion, evaluation, and deployment references
+Grep: "model_uri|model_id|model_version|revision|digest|sha256|evaluation_run|model_card|approval" in **/*.{py,yaml,yml,json,md}
+Grep: "latest|main|prod|staging|canary|shadow|rollback|deploy" in **/*.{py,yaml,yml,json,md,sh}
+Grep: "lora|qlora|peft|adapter|base_model|tokenizer" in **/*.{py,yaml,yml,json,md}
+
+# Find registry and storage paths that may be mutable
+Grep: "s3://|gs://|az://|hf://|huggingface://|registry" in **/*.{py,yaml,yml,json,md,sh}
+```
+
+**Promotion evidence gates:**
+
+| Gate | Evidence Required | Finding Trigger |
+|---|---|---|
+| MODEL-PROMO-01 | Evaluation report includes immutable model identity: digest, commit SHA, object version, provider version ID, or signed attestation. | Evaluation names only a model family, repo, or mutable alias. |
+| MODEL-PROMO-02 | Approval record binds artifact identity, evaluation run ID, model card version, policy decision, and approver authority. | Approval ticket names the model but not the exact evaluated artifact. |
+| MODEL-PROMO-03 | Deployment manifest references immutable model, adapter, tokenizer, container, and serving-code identities. | Deployment pulls `main`, `latest`, unversioned paths, or current provider defaults. |
+| MODEL-PROMO-04 | Promotion path enforces registry immutability, signature or attestation verification, and restricted write access. | A user with training or registry access can replace a released artifact after approval. |
+| MODEL-PROMO-05 | Canary, shadow, staging, production, and regional deployments record served artifact identity and rollout status. | Environment-specific deployments cannot be tied back to the approved artifact. |
+| MODEL-PROMO-06 | Adapter releases bind base model digest, adapter digest, tokenizer/config version, and merge procedure. | LoRA/adapter approval omits base model identity or merged artifact verification. |
+| MODEL-PROMO-07 | Rollback targets have checksum/signature verification, evaluation status, model card version, and known-risk review. | Rollback uses aliases or old bucket prefixes without re-verifying the artifact. |
+| MODEL-PROMO-08 | Post-deploy verification confirms the runtime served identity matches the approved deploy manifest. | Startup logs, registry metadata, or endpoint metadata are not checked after release. |
+
+**What constitutes a finding:**
+
+| Condition | Severity |
+|---|---|
+| Production deploys a model artifact that cannot be tied to the evaluated artifact | High |
+| Deployment uses mutable model or container references for production inference | High |
+| Approval is not bound to digest/revision, evaluation run, and model card version | High |
+| LoRA/adapter release omits base model or merged artifact identity | High |
+| Rollback target is unverified or lacks current risk/evaluation status | Medium |
+| Environment-specific rollout identity is missing for canary or shadow deployments | Medium |
+
+---
+
 ### Step 4 -- Inference Dependency Review
 
 Assess the security of libraries, frameworks, and runtime dependencies used in the model serving path.
@@ -382,10 +434,16 @@ Assess whether architectural and procedural controls exist to detect model backd
 |---|---|---|---|---|---|
 | [name] | [source] | [format] | [Yes/No] | [Yes/No] | [Complete/Partial/Missing] |
 
+## Promotion and Deployment Binding
+
+| Environment | Evaluated Artifact | Approval / Eval Run | Deploy Manifest Identity | Served Identity Verified | Rollback Verified |
+|---|---|---|---|---|---|
+| [prod/staging/canary] | [digest/revision/object version] | [ticket/run/model card] | [model/container/adapter identity] | [Yes/No + evidence] | [Yes/No + evidence] |
+
 ## Findings
 
 ### Finding [N]: [Title]
-- **Category:** [Provenance | Training Data | Fine-Tuning Pipeline | Inference Dependency | Model Card | Backdoor Detection]
+- **Category:** [Provenance | Promotion Binding | Training Data | Fine-Tuning Pipeline | Inference Dependency | Model Card | Backdoor Detection]
 - **Severity:** [Critical | High | Medium | Low | Informational]
 - **OWASP LLM Category:** LLM03:2025 -- Supply Chain Vulnerabilities
 - **MITRE ATLAS Technique:** [technique ID and name]
@@ -401,6 +459,7 @@ Assess whether architectural and procedural controls exist to detect model backd
 | Domain | Current State | Target State | Gap Severity |
 |---|---|---|---|
 | Model provenance | [description] | [recommendation] | [severity] |
+| Promotion and deployment binding | [description] | [recommendation] | [severity] |
 | Training data lineage | [description] | [recommendation] | [severity] |
 | Fine-tuning pipeline | [description] | [recommendation] | [severity] |
 | Inference dependencies | [description] | [recommendation] | [severity] |

skills/ai-security/model-supply-chain/tests/benign/promotion-artifact-identity-bound.json

@@ -0,0 +1,46 @@
+{
+  "case": "promotion_artifact_identity_bound",
+  "skill": "model-supply-chain",
+  "expected_result": "pass",
+  "model": {
+    "name": "support-intent-classifier",
+    "source": "huggingface://acme/support-intent-classifier",
+    "revision": "8f44a5e2c9f2a97b4c12e48c8a94a5f0a6d4b331",
+    "artifact_sha256": "f8a0d7ef56d4f9b5b5d682d0c6a5c35f86d45a31d09c3b2b7f3f536b32b6f4c1",
+    "model_card_version": "2026-06-03"
+  },
+  "evaluation": {
+    "run_id": "eval-2026-06-05-0412",
+    "dataset_snapshot": "customer-support-eval@sha256:80ab0c1e",
+    "artifact_sha256": "f8a0d7ef56d4f9b5b5d682d0c6a5c35f86d45a31d09c3b2b7f3f536b32b6f4c1",
+    "status": "passed"
+  },
+  "approval": {
+    "ticket": "AI-REL-4721",
+    "approved_artifact_sha256": "f8a0d7ef56d4f9b5b5d682d0c6a5c35f86d45a31d09c3b2b7f3f536b32b6f4c1",
+    "approved_eval_run": "eval-2026-06-05-0412",
+    "approver_role": "ml-risk-owner"
+  },
+  "deployment": {
+    "environment": "production",
+    "manifest_model_digest": "sha256:f8a0d7ef56d4f9b5b5d682d0c6a5c35f86d45a31d09c3b2b7f3f536b32b6f4c1",
+    "serving_image_digest": "sha256:dcf3b6a186690f6ab7616e39672ad8625d710e9e3f4b5077b5e9c9196b65d203",
+    "served_identity_log": "startup-2026-06-06T02:10:21Z",
+    "served_identity_verified": true
+  },
+  "rollback": {
+    "target_digest": "sha256:429a27f74fe9fbf9c5f83bbf6d82c9466a6c0df3bb8d1c43f9e9e5fe6f24a11a",
+    "signature_verified": true,
+    "evaluation_status": "passed",
+    "known_risk_review": "current"
+  },
+  "covered_gates": [
+    "MODEL-PROMO-01",
+    "MODEL-PROMO-02",
+    "MODEL-PROMO-03",
+    "MODEL-PROMO-04",
+    "MODEL-PROMO-05",
+    "MODEL-PROMO-07",
+    "MODEL-PROMO-08"
+  ]
+}

skills/ai-security/model-supply-chain/tests/vulnerable/mutable-promotion-alias-overcredited.json

@@ -0,0 +1,66 @@
+{
+  "case": "mutable_promotion_alias_overcredited",
+  "skill": "model-supply-chain",
+  "expected_result": "finding",
+  "model": {
+    "name": "fraud-risk-ranker",
+    "source": "huggingface://finance/fraud-risk-ranker",
+    "revision": "main",
+    "artifact_sha256": null,
+    "model_card_version": "latest"
+  },
+  "evaluation": {
+    "run_id": "eval-2026-06-04-2210",
+    "reported_model": "finance/fraud-risk-ranker",
+    "artifact_sha256": null,
+    "status": "passed"
+  },
+  "approval": {
+    "ticket": "AI-REL-4777",
+    "approved_model_family": "fraud-risk-ranker",
+    "approved_artifact_sha256": null,
+    "approved_eval_run": null,
+    "approver_role": "product-owner"
+  },
+  "deployment": {
+    "environment": "production",
+    "manifest_model_uri": "s3://models/fraud-risk-ranker/latest",
+    "serving_image": "registry.example.com/ml/fraud-risk:latest",
+    "adapter_uri": "s3://models/fraud-risk-ranker/adapters/current",
+    "base_model_digest": null,
+    "adapter_digest": null,
+    "served_identity_verified": false
+  },
+  "rollback": {
+    "target": "s3://models/fraud-risk-ranker/previous",
+    "signature_verified": false,
+    "evaluation_status": "unknown",
+    "known_risk_review": "missing"
+  },
+  "expected_findings": [
+    {
+      "gate": "MODEL-PROMO-01",
+      "reason": "Evaluation does not identify the immutable artifact that was tested."
+    },
+    {
+      "gate": "MODEL-PROMO-02",
+      "reason": "Approval is bound to a model family instead of digest, evaluation run, and model card version."
+    },
+    {
+      "gate": "MODEL-PROMO-03",
+      "reason": "Deployment pulls mutable model and container aliases."
+    },
+    {
+      "gate": "MODEL-PROMO-06",
+      "reason": "Adapter release omits both base model and adapter digests."
+    },
+    {
+      "gate": "MODEL-PROMO-07",
+      "reason": "Rollback target is an alias with no signature, evaluation, or risk review evidence."
+    },
+    {
+      "gate": "MODEL-PROMO-08",
+      "reason": "Runtime served identity was not verified against the approved manifest."
+    }
+  ]
+}