[BUG] missing package-level annotation prints wrong pkg name in EC report

[dheerajodha]

Describe the Bug

While validating an image against a policy that's missing the package-level annotation, the EC output prints the rule as deny.<rulename> instead of <packagename>.<rulename>

Steps to Reproduce

Provide a detailed step by step description of how to reproduce the bug.

1. Run command ec validate image --policy policy.yaml --public-key cosign-m01.pub --image $IMAGE --output text --ignore-rekor --info --show-successes --debug
2. Use this image and Policy config:

$ echo $IMAGE
quay.io/redhat-user-workloads/rhn-support-djodha-tenant/the-mentalist-quiz/the-mentalist-quiz@sha256:0cfce998e1173d8f6147b8ac53626b86715413ad24c2fc413d03882ee0568cbe

$ cat policy.yaml 
sources:
  - policy:
      - github.com/enterprise-contract/ec-policies//policy/lib
      - github.com/dheerajodha/the-mentalist-quiz.git//policies?ref=add-custom-policy

3. Use this Rule definition:

package mentalist.policies.verify_uri

import rego.v1

import data.lib

# METADATA
# title: Verify URI count
# description: Confirm there are entries (other than 1) in the predicate.materials
#   array of the attestation that contains the specific uri as seen in the below query
# custom:
#   short_name: verify_uri
#   failure_msg: Unexpected count of URI prefix matches
#   solution: Fix it
deny contains result if {
	match := [material |
		some material in materials
		material.uri == "git+https://github.com/dheerajodha/the-mentalist-quiz.git"
	]
	print(match)
	count(match) == 1
	result := lib.result_helper(rego.metadata.chain(), [])
}

materials contains material if {
	some attestation in input.attestations
		some material in attestation.statement.predicate.materials
			material.uri
			material.digest.sha1
}

Expected Behavior

Output:

Results:
✕ [Violation] verify_uri.verify_uri
  ImageRef: quay.io/redhat-user-workloads/rhn-support-djodha-tenant/the-mentalist-quiz/the-mentalist-quiz@sha256:0cfce998e1173d8f6147b8ac53626b86715413ad24c2fc413d03882ee0568cbe
  Reason: Unexpected count of URI prefix matches

Actual Behavior

Output:

Results:
✕ [Violation] deny.verify_uri
  ImageRef: quay.io/redhat-user-workloads/rhn-support-djodha-tenant/the-mentalist-quiz/the-mentalist-quiz@sha256:0cfce998e1173d8f6147b8ac53626b86715413ad24c2fc413d03882ee0568cbe
  Reason: Unexpected count of URI prefix matches

Screenshots or Terminal Output

Click me to see command + full-terminal-output

$ ec validate image --policy policy.yaml --public-key cosign-m01.pub --image $IMAGE --output text --ignore-rekor --info --show-successes --debug DEBU[0000] input.go:134 DetermineInputSpec Generating application snapshot from image reference quay.io/redhat-user-workloads/rhn-support-djodha-tenant/the-mentalist-quiz/the-mentalist-quiz@sha256:0cfce998e1173d8f6147b8ac53626b86715413ad24c2fc413d03882ee0568cbe DEBU[0002] input.go:243 expandImageIndex Snap component after expanding the image index is [{Unnamed quay.io/redhat-user-workloads/rhn-support-djodha-tenant/the-mentalist-quiz/the-mentalist-quiz@sha256:0cfce998e1173d8f6147b8ac53626b86715413ad24c2fc413d03882ee0568cbe {{}}}] DEBU[0002] helpers.go:58 GetPolicyConfig Loading policy.yaml as policy configuration DEBU[0002] helpers.go:79 ReadFile Loaded policy.yaml DEBU[0002] policy.go:309 loadPolicy Read EnterpriseContractPolicy as YAML DEBU[0002] policy.go:314 loadPolicy Unable to parse EnterpriseContractPolicy from "sources:\n - policy:\n - github.com/enterprise-contract/ec-policies//policy/lib\n - github.com/dheerajodha/the-mentalist-quiz.git//policies?ref=add-custom-policy\n" DEBU[0002] policy.go:315 loadPolicy Attempting to parse as EnterpriseContractPolicySpec DEBU[0002] policy.go:255 NewPolicy Updated public key in policy to "cosign-m01.pub" DEBU[0002] policy.go:394 parseEffectiveTime Chosen to use effective time of `now`, using current time 2024-08-29T19:35:46Z DEBU[0002] policy.go:428 checkOpts Using long-lived key workflow DEBU[0002] image.go:297 func2 Fetching policy source group '' DEBU[0002] image.go:305 func2 policySource: &source.PolicyUrl{Url:"github.com/enterprise-contract/ec-policies//policy/lib", Kind:"policy"} DEBU[0002] image.go:305 func2 policySource: &source.PolicyUrl{Url:"github.com/dheerajodha/the-mentalist-quiz.git//policies?ref=add-custom-policy", Kind:"policy"} DEBU[0002] conftest_evaluator.go:308 NewConftestEvaluatorWithNamespace Created work dir /tmp/ec-work-836787321 DEBU[0002] policy.go:380 EffectiveTime Using effective time: 2024-08-29T19:35:46Z DEBU[0002] conftest_evaluator.go:735 createConfigJSON Writing config data to /tmp/ec-work-836787321/data/config.json: "{\n \"config\": {\n \"default_sigstore_opts\": {\n \"certificate_identity\": \"\",\n \"certificate_identity_regexp\": \"\",\n \"certificate_oidc_issuer\": \"\",\n \"certificate_oidc_issuer_regexp\": \"\",\n \"ignore_rekor\": true,\n \"public_key\": \"-----BEGIN PUBLIC KEY-----\\nMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEWVUppvU1x8t866CQQSXbKpojoaTq\\nimMnVnZ31e2ubZHKL1LdfgPG2gHIPeSeouTa8upOz9W+xxBFnA0X515Nsw==\\n-----END PUBLIC KEY-----\\n\",\n \"rekor_url\": \"\"\n },\n \"policy\": {\n \"when_ns\": 1724960146129598289\n }\n }\n}" DEBU[0002] conftest_evaluator.go:958 strictCapabilities Network access from rego policies disabled DEBU[0002] conftest_evaluator.go:974 strictCapabilities Access to some rego built-in functions disabled: [http.send net.lookup_ip_addr opa.runtime] DEBU[0002] conftest_evaluator.go:780 createCapabilitiesFile Capabilities file written to /tmp/ec-work-836787321/capabilities.json DEBU[0002] conftest_evaluator.go:318 NewConftestEvaluatorWithNamespace Conftest test runner created DEBU[0002] image.go:323 1 Starting worker 5 DEBU[0002] image.go:323 1 Starting worker 1 DEBU[0002] image.go:358 1 Done with worker 1 DEBU[0002] image.go:325 1 Worker 5 got a component "quay.io/redhat-user-workloads/rhn-support-djodha-tenant/the-mentalist-quiz/the-mentalist-quiz@sha256:0cfce998e1173d8f6147b8ac53626b86715413ad24c2fc413d03882ee0568cbe" DEBU[0002] image.go:323 1 Starting worker 4 DEBU[0002] image.go:358 1 Done with worker 4 DEBU[0002] image.go:323 1 Starting worker 0 DEBU[0002] image.go:323 1 Starting worker 2 DEBU[0002] image.go:358 1 Done with worker 0 DEBU[0002] validate.go:39 ValidateImage Validating image quay.io/redhat-user-workloads/rhn-support-djodha-tenant/the-mentalist-quiz/the-mentalist-quiz@sha256:0cfce998e1173d8f6147b8ac53626b86715413ad24c2fc413d03882ee0568cbe DEBU[0002] application_snapshot_image.go:107 SetImageURL Parsed image url quay.io/redhat-user-workloads/rhn-support-djodha-tenant/the-mentalist-quiz/the-mentalist-quiz@sha256:0cfce998e1173d8f6147b8ac53626b86715413ad24c2fc413d03882ee0568cbe DEBU[0002] image.go:358 1 Done with worker 2 DEBU[0002] image.go:323 1 Starting worker 3 DEBU[0002] image.go:358 1 Done with worker 3 DEBU[0004] application_snapshot_image.go:97 ValidateImageAccess Resp: &{MediaType:application/vnd.oci.image.manifest.v1+json Size:1996 Digest:sha256:0cfce998e1173d8f6147b8ac53626b86715413ad24c2fc413d03882ee0568cbe Data:[] URLs:[] Annotations:map[] Platform: ArtifactType:} DEBU[0004] output.go:104 SetImageAccessibleCheckFromError Image URL is accessible DEBU[0004] validate.go:153 resolveAndSetImageUrl Resolved image to quay.io/redhat-user-workloads/rhn-support-djodha-tenant/the-mentalist-quiz/the-mentalist-quiz@sha256:0cfce998e1173d8f6147b8ac53626b86715413ad24c2fc413d03882ee0568cbe DEBU[0004] application_snapshot_image.go:107 SetImageURL Parsed image url quay.io/redhat-user-workloads/rhn-support-djodha-tenant/the-mentalist-quiz/the-mentalist-quiz@sha256:0cfce998e1173d8f6147b8ac53626b86715413ad24c2fc413d03882ee0568cbe DEBU[0005] client.go:83 initCache using "/home/djodha/.cache/ec/images" directory to store image cache DEBU[0014] output.go:129 SetImageSignatureCheckFromError Image signature check passed DEBU[0015] application_snapshot_image.go:179 ValidateAttestationSignature Found attestation with predicateType: https://slsa.dev/provenance/v0.2 DEBU[0015] output.go:154 SetAttestationSignatureCheckFromError Attestation signature check passed DEBU[0015] application_snapshot_image.go:223 ValidateAttestationSyntax Attempting to validate an attestation with predicateType https://slsa.dev/provenance/v0.2 DEBU[0015] application_snapshot_image.go:238 ValidateAttestationSyntax Statement schema was validated successfully against the https://slsa.dev/provenance/v0.2 schema DEBU[0015] output.go:179 SetAttestationSyntaxCheckFromError Attestation syntax check passed DEBU[0015] validate.go:214 determineAttestationTime Determined attestation time: 2024-08-27T17:30:47Z DEBU[0015] validate.go:89 ValidateImage Found 1 attestations DEBU[0015] application_snapshot_image.go:336 WriteInputFile Attempting to write 1 attestations to input file DEBU[0015] application_snapshot_image.go:371 WriteInputFile Created dir /tmp/ecp_input.038800772 DEBU[0015] application_snapshot_image.go:390 WriteInputFile Done preparing input file: /tmp/ecp_input.038800772/input.json DEBU[0015] source.go:94 func1 Download cache miss: github.com/enterprise-contract/ec-policies//policy/lib DEBU[0015] source.go:96 func1 Downloading policy files from source url github.com/enterprise-contract/ec-policies//policy/lib to destination /tmp/ec-work-836787321/policy/1253694d3 DEBU[0015] downloader.go:63 Download Downloading github.com/enterprise-contract/ec-policies//policy/lib to /tmp/ec-work-836787321/policy/1253694d3 DEBU[0040] source.go:94 func1 Download cache miss: github.com/dheerajodha/the-mentalist-quiz.git//policies?ref=add-custom-policy DEBU[0040] source.go:96 func1 Downloading policy files from source url github.com/dheerajodha/the-mentalist-quiz.git//policies?ref=add-custom-policy to destination /tmp/ec-work-836787321/policy/b909ff96d DEBU[0040] downloader.go:63 Download Downloading github.com/dheerajodha/the-mentalist-quiz.git//policies?ref=add-custom-policy to /tmp/ec-work-836787321/policy/b909ff96d DEBU[0042] conftest_evaluator.go:437 Evaluate runner: &evaluator.conftestRunner{TestRunner:runner.TestRunner{Trace:false, Strict:false, Capabilities:"/tmp/ec-work-836787321/capabilities.json", Policy:[]string{"/tmp/ec-work-836787321/policy"}, Data:[]string{"/tmp/ec-work-836787321/data"}, Update:[]string(nil), Ignore:"", Parser:"", Namespace:[]string(nil), AllNamespaces:true, FailOnWarn:false, NoColor:false, NoFail:true, SuppressExceptions:false, ShowBuiltinErrors:false, Combine:false, Quiet:false, Output:"json"}} DEBU[0042] conftest_evaluator.go:438 Evaluate inputs: []string{"/tmp/ecp_input.038800772/input.json"} DEBU[0042] conftest_evaluator.go:228 Run [data.mentalist.policies.verify_uri.deny] /tmp/ec-work-836787321/policy/b909ff96d/verify_uri.rego:20: [{"digest": {"sha1": "912a9b762f92427d7d39af4b866737514a5c5dab"}, "uri": "git+https://github.com/dheerajodha/the-mentalist-quiz.git"}] DEBU[0042] policy.go:380 EffectiveTime Using effective time: 2024-08-29T19:35:46Z DEBU[0042] conftest_evaluator.go:456 Evaluate Evaluation result at 0: evaluator.Outcome{FileName:"/tmp/ecp_input.038800772/input.json", Namespace:"lib.tkn.recorded_att_test", Successes:[]evaluator.Result{}, Skipped:[]evaluator.Result(nil), Warnings:[]evaluator.Result(nil), Failures:[]evaluator.Result(nil), Exceptions:[]evaluator.Result(nil)} DEBU[0042] conftest_evaluator.go:456 Evaluate Evaluation result at 1: evaluator.Outcome{FileName:"/tmp/ecp_input.038800772/input.json", Namespace:"lib.refs", Successes:[]evaluator.Result{}, Skipped:[]evaluator.Result(nil), Warnings:[]evaluator.Result(nil), Failures:[]evaluator.Result(nil), Exceptions:[]evaluator.Result(nil)} DEBU[0042] conftest_evaluator.go:456 Evaluate Evaluation result at 2: evaluator.Outcome{FileName:"/tmp/ecp_input.038800772/input.json", Namespace:"lib", Successes:[]evaluator.Result{}, Skipped:[]evaluator.Result(nil), Warnings:[]evaluator.Result(nil), Failures:[]evaluator.Result(nil), Exceptions:[]evaluator.Result(nil)} DEBU[0042] conftest_evaluator.go:456 Evaluate Evaluation result at 3: evaluator.Outcome{FileName:"/tmp/ecp_input.038800772/input.json", Namespace:"lib.arrays", Successes:[]evaluator.Result{}, Skipped:[]evaluator.Result(nil), Warnings:[]evaluator.Result(nil), Failures:[]evaluator.Result(nil), Exceptions:[]evaluator.Result(nil)} DEBU[0042] conftest_evaluator.go:456 Evaluate Evaluation result at 4: evaluator.Outcome{FileName:"/tmp/ecp_input.038800772/input.json", Namespace:"policy.task.k8s_test", Successes:[]evaluator.Result{}, Skipped:[]evaluator.Result(nil), Warnings:[]evaluator.Result(nil), Failures:[]evaluator.Result(nil), Exceptions:[]evaluator.Result(nil)} DEBU[0042] conftest_evaluator.go:456 Evaluate Evaluation result at 5: evaluator.Outcome{FileName:"/tmp/ecp_input.038800772/input.json", Namespace:"lib.bundles", Successes:[]evaluator.Result{}, Skipped:[]evaluator.Result(nil), Warnings:[]evaluator.Result(nil), Failures:[]evaluator.Result(nil), Exceptions:[]evaluator.Result(nil)} DEBU[0042] conftest_evaluator.go:456 Evaluate Evaluation result at 6: evaluator.Outcome{FileName:"/tmp/ecp_input.038800772/input.json", Namespace:"lib_test", Successes:[]evaluator.Result{}, Skipped:[]evaluator.Result(nil), Warnings:[]evaluator.Result(nil), Failures:[]evaluator.Result(nil), Exceptions:[]evaluator.Result(nil)} DEBU[0042] conftest_evaluator.go:456 Evaluate Evaluation result at 7: evaluator.Outcome{FileName:"/tmp/ecp_input.038800772/input.json", Namespace:"lib.tkn", Successes:[]evaluator.Result{}, Skipped:[]evaluator.Result(nil), Warnings:[]evaluator.Result(nil), Failures:[]evaluator.Result(nil), Exceptions:[]evaluator.Result(nil)} DEBU[0042] conftest_evaluator.go:456 Evaluate Evaluation result at 8: evaluator.Outcome{FileName:"/tmp/ecp_input.038800772/input.json", Namespace:"lib.bundles_test", Successes:[]evaluator.Result{}, Skipped:[]evaluator.Result(nil), Warnings:[]evaluator.Result(nil), Failures:[]evaluator.Result(nil), Exceptions:[]evaluator.Result(nil)} DEBU[0042] conftest_evaluator.go:456 Evaluate Evaluation result at 9: evaluator.Outcome{FileName:"/tmp/ecp_input.038800772/input.json", Namespace:"lib.time", Successes:[]evaluator.Result{}, Skipped:[]evaluator.Result(nil), Warnings:[]evaluator.Result(nil), Failures:[]evaluator.Result(nil), Exceptions:[]evaluator.Result(nil)} DEBU[0042] conftest_evaluator.go:456 Evaluate Evaluation result at 10: evaluator.Outcome{FileName:"/tmp/ecp_input.038800772/input.json", Namespace:"mentalist.policies.verify_uri", Successes:[]evaluator.Result{}, Skipped:[]evaluator.Result(nil), Warnings:[]evaluator.Result(nil), Failures:[]evaluator.Result{evaluator.Result{Message:"Unexpected count of URI prefix matches", Metadata:map[string]interface {}{"code":"deny.verify_uri", "effective_on":"2022-01-01T00:00:00Z"}, Outputs:[]string(nil)}}, Exceptions:[]evaluator.Result(nil)} DEBU[0042] conftest_evaluator.go:456 Evaluate Evaluation result at 11: evaluator.Outcome{FileName:"/tmp/ecp_input.038800772/input.json", Namespace:"lib.image", Successes:[]evaluator.Result{}, Skipped:[]evaluator.Result(nil), Warnings:[]evaluator.Result(nil), Failures:[]evaluator.Result(nil), Exceptions:[]evaluator.Result(nil)} DEBU[0042] conftest_evaluator.go:456 Evaluate Evaluation result at 12: evaluator.Outcome{FileName:"/tmp/ecp_input.038800772/input.json", Namespace:"lib.sbom_test", Successes:[]evaluator.Result{}, Skipped:[]evaluator.Result(nil), Warnings:[]evaluator.Result(nil), Failures:[]evaluator.Result(nil), Exceptions:[]evaluator.Result(nil)} DEBU[0042] conftest_evaluator.go:456 Evaluate Evaluation result at 13: evaluator.Outcome{FileName:"/tmp/ecp_input.038800772/input.json", Namespace:"lib.time_test", Successes:[]evaluator.Result{}, Skipped:[]evaluator.Result(nil), Warnings:[]evaluator.Result(nil), Failures:[]evaluator.Result(nil), Exceptions:[]evaluator.Result(nil)} DEBU[0042] conftest_evaluator.go:456 Evaluate Evaluation result at 14: evaluator.Outcome{FileName:"/tmp/ecp_input.038800772/input.json", Namespace:"lib.arrays_test", Successes:[]evaluator.Result{}, Skipped:[]evaluator.Result(nil), Warnings:[]evaluator.Result(nil), Failures:[]evaluator.Result(nil), Exceptions:[]evaluator.Result(nil)} DEBU[0042] conftest_evaluator.go:456 Evaluate Evaluation result at 15: evaluator.Outcome{FileName:"/tmp/ecp_input.038800772/input.json", Namespace:"lib.tkn_test", Successes:[]evaluator.Result{}, Skipped:[]evaluator.Result(nil), Warnings:[]evaluator.Result(nil), Failures:[]evaluator.Result(nil), Exceptions:[]evaluator.Result(nil)} DEBU[0042] conftest_evaluator.go:456 Evaluate Evaluation result at 16: evaluator.Outcome{FileName:"/tmp/ecp_input.038800772/input.json", Namespace:"lib.image_test", Successes:[]evaluator.Result{}, Skipped:[]evaluator.Result(nil), Warnings:[]evaluator.Result(nil), Failures:[]evaluator.Result(nil), Exceptions:[]evaluator.Result(nil)} DEBU[0042] conftest_evaluator.go:456 Evaluate Evaluation result at 17: evaluator.Outcome{FileName:"/tmp/ecp_input.038800772/input.json", Namespace:"lib.tkn.pipeline_test", Successes:[]evaluator.Result{}, Skipped:[]evaluator.Result(nil), Warnings:[]evaluator.Result(nil), Failures:[]evaluator.Result(nil), Exceptions:[]evaluator.Result(nil)} DEBU[0042] conftest_evaluator.go:456 Evaluate Evaluation result at 18: evaluator.Outcome{FileName:"/tmp/ecp_input.038800772/input.json", Namespace:"lib.sbom", Successes:[]evaluator.Result{}, Skipped:[]evaluator.Result(nil), Warnings:[]evaluator.Result(nil), Failures:[]evaluator.Result(nil), Exceptions:[]evaluator.Result(nil)} DEBU[0042] conftest_evaluator.go:456 Evaluate Evaluation result at 19: evaluator.Outcome{FileName:"/tmp/ecp_input.038800772/input.json", Namespace:"lib.refs_test", Successes:[]evaluator.Result{}, Skipped:[]evaluator.Result(nil), Warnings:[]evaluator.Result(nil), Failures:[]evaluator.Result(nil), Exceptions:[]evaluator.Result(nil)} DEBU[0042] conftest_evaluator.go:456 Evaluate Evaluation result at 20: evaluator.Outcome{FileName:"/tmp/ecp_input.038800772/input.json", Namespace:"lib.k8s", Successes:[]evaluator.Result{}, Skipped:[]evaluator.Result(nil), Warnings:[]evaluator.Result(nil), Failures:[]evaluator.Result(nil), Exceptions:[]evaluator.Result(nil)} DEBU[0042] validate.go:119 ValidateImage

Running conftest policy check

DEBU[0042] validate.go:131 ValidateImage Conftest policy check complete
DEBU[0042] image.go:358 1 Done with worker 5
DEBU[0042] policy.go:380 EffectiveTime Using effective time: 2024-08-29T19:35:46Z
Success: false
Result: FAILURE
Violations: 1, Warnings: 0, Successes: 4
Component: Unnamed
ImageRef: quay.io/redhat-user-workloads/rhn-support-djodha-tenant/the-mentalist-quiz/the-mentalist-quiz@sha256:0cfce998e1173d8f6147b8ac53626b86715413ad24c2fc413d03882ee0568cbe

Results:
✕ [Violation] deny.verify_uri
ImageRef: quay.io/redhat-user-workloads/rhn-support-djodha-tenant/the-mentalist-quiz/the-mentalist-quiz@sha256:0cfce998e1173d8f6147b8ac53626b86715413ad24c2fc413d03882ee0568cbe
Reason: Unexpected count of URI prefix matches

✓ [Success] builtin.attestation.signature_check
ImageRef: quay.io/redhat-user-workloads/rhn-support-djodha-tenant/the-mentalist-quiz/the-mentalist-quiz@sha256:0cfce998e1173d8f6147b8ac53626b86715413ad24c2fc413d03882ee0568cbe
Title: Attestation signature check passed
Description: The attestation signature matches available signing materials.

✓ [Success] builtin.attestation.syntax_check
ImageRef: quay.io/redhat-user-workloads/rhn-support-djodha-tenant/the-mentalist-quiz/the-mentalist-quiz@sha256:0cfce998e1173d8f6147b8ac53626b86715413ad24c2fc413d03882ee0568cbe
Title: Attestation syntax check passed
Description: The attestation has correct syntax.

✓ [Success] builtin.image.signature_check
ImageRef: quay.io/redhat-user-workloads/rhn-support-djodha-tenant/the-mentalist-quiz/the-mentalist-quiz@sha256:0cfce998e1173d8f6147b8ac53626b86715413ad24c2fc413d03882ee0568cbe
Title: Image signature check passed
Description: The image signature matches available signing materials.

✓ [Success] mentalist.policies.verify_uri.verify_uri
ImageRef: quay.io/redhat-user-workloads/rhn-support-djodha-tenant/the-mentalist-quiz/the-mentalist-quiz@sha256:0cfce998e1173d8f6147b8ac53626b86715413ad24c2fc413d03882ee0568cbe
Title: Verify URI count
Description: Confirm there are entries (other than 1) in the predicate.materials array of the attestation that contains the
specific uri as seen in the below query

Error: success criteria not met

Environment Details

• Operating System: Fedora Linux 40
• ec CLI Version, run ec version --short: v0.5.115
• Shell:
• Additional context:

Possible Solution

Add the package-level annotation, and EC should print the correct package name in the output.