Enhance multi-source evaluation support in Layer 4/5

[jpower432]

Objective

Currently there is a location in the EvaluationPlan to define information about the assessment tool in the metadata under author. If we update the EvalautionPlan to allow tools to be attached to certain assessment procedures, we can enable a workflow like that supports multi-source Evaluations.

• Layer 4 EvaluationLog preserves granular outcomes from individual tools/executors to maintain integrity.
• Layer 5 aggregates multiple EvaluationLogs from different tools, deconflicts evidence across tools using the granular results, and identifies findings for enforcement.
• Layer 4 EvaluationPlan guides this aggregation

Diagram

flowchart TD
    subgraph L1["Layer 1"]
        G1["Guidance 1<br/>(e.g., NIST CSF)"]
        G2["Guidance 2<br/>(e.g., ISO 27001)"]
        G3["Guidance 3<br/>(e.g., PCI DSS)"]
        G4["...<br/>More Guidances"]
    end
    subgraph L2["Layer 2"]
        C1["Control Catalog 1<br/>(e.g., CIS Benchmarks)"]
        C2["Control Catalog 2<br/>(e.g., CCC)"]
        C3["Control Catalog 3<br/>(e.g., OSPS)"]
        C4["...<br/>More Controls"]
    end
    subgraph L3["Layer 3"]
        P1["Policy Document 1"]
        P2["Policy Document 2"]
        P3["...<br/>More Policies"]
    end

    subgraph L4["Layer 4"]
        EP["EvaluationPlan"]
        EL1["Evaluation Log 1"]
        EL2["Evaluation Log 2"]
        EL3["Evaluation Log 3"]
        EL4["...<br/>More Logs"]
    end

    subgraph L5["Layer 5"]
        F["Findings"]
        EA["EnforcementAction"]
        O["Layer5 Library"]
    end

    subgraph L6["Layer 6: Audit"]
        A["Audit"]
    end


    G1 --> P1
    G2 --> P1
    G3 --> P2
    G4 --> P3
    
    C1 --> P1
    C2 --> P1
    C3 --> P2
    C4 --> P3

    P1 --> EP
    P2 --> EP
    P3 --> EP

    EP --> EL1
    EP --> EL2
    EP --> EL3
    EP --> EL4

    EL1 -->|"technical status"|O
    EL2 -->|"technical status"|O
    EL3 -->|"technical status"|O
    EL4 -->|"technical status"|O

    O -->|"policy context"| F

    EP -->|"aggregation instructions"|O

    P1 -->|"enforcement method"| EA
    P2 -->|"enforcement method"| EA
    P3 -->|"enforcement method"| EA
    F --> EA

    L1 --> A
    L2 --> A
    L3 --> A
    L4 --> A
    L5 --> A

Loading

Criteria

• Add assessment tool to EvaluationPlan and attach them to N assessment procedures
• Add a section in an EvaluationPlan for instructions for tool rank or conflict resolution between tools
• In feat: adds initial schema for Layer 5 (for discussion) #165
  • The Layer 5 library accepts the EvaluationPlan and N EvaluationLogs (one per tool) the output in Findings
  • Explicitly link the Layer 3 Policy
  • Should add a Run when the Enforcement Action is performed

[jpower432]

@eddie-knight I worked on capturing the Layer 4/5 discussion outcomes here. Please let me know if I missed anything.