Skip to content

Distinguish simplifying-assumptions (adequacy) from threshold-criteria (sufficiency) in attestation vocabulary #30

@mzargham

Description

@mzargham

Surfaced by

UAT walkthrough of the engineer skill against the ADCS arc, while
authoring REQ-001's rdfs:comment. The engineer playing the
walkthrough drew a sharp ontological distinction that the current
attestation vocabulary doesn't capture structurally:

"Model simplifying assumptions and justification fall under the
adequacy attestation. The specific criteria which tell us which
thresholds are good enough fall under sufficiency."

That's a clean partition. The requirement statement holds the bare
claim (with thresholds: 0.1°, 120s); the adequacy attestation holds
the "why is this model faithful enough" reasoning (which simplifying
assumptions were taken; which physics is excluded); the sufficiency
attestation holds the "why is this evidence enough to establish the
thresholds" reasoning (which slew amplitudes were tested; which
disturbance magnitudes; which evaluation method).

What's missing in v0.1

Today both kinds of justification land in rdfs:comment free text on
the attestation. Concretely:

<urn:rtm:attest/adequacy-ev-proof-req-001>
    a rtm:AdequacyAttestation ;
    rtm:approvedBy <...> ;
    rtm:status rtm:pass ;
    rtm:appliesTo <https://rtm.example/adcs/EV-PROOF-REQ-001> ;
    rdfs:comment "Rigid-body 6-DOF model with linearized reaction-wheel
                  actuator dynamics; gravity gradient only; no solar
                  pressure; no flex modes. Adequate because [...]." .

The comment is a single unstructured block. An audit can't:

  • Enumerate the simplifying assumptions an adequacy claim relies on
    (SPARQL: "show me every adequacy attestation that excluded solar
    pressure").
  • Query whether a specific assumption was made consistently across
    related attestations.
  • Detect when a NEW source of evidence (e.g., a re-run sim with
    wider envelope) violates a prior adequacy assumption — the prior
    was "we simplify to gravity gradient only", but the re-run added
    solar pressure, so the prior adequacy attestation should
    AUTOMATICALLY be flagged as no-longer-applicable, not just
    re-judged manually.

For the sufficiency side, analogously:

<urn:rtm:attest/sufficiency-ev-sim-req-001>
    a rtm:SufficiencyAttestation ;
    rtm:appliesTo <https://rtm.example/adcs/EV-SIM-REQ-001> ;
    rtm:status rtm:fail ;
    rdfs:comment "The simulation sweeps disturbance magnitudes only
                  up to 1× the nominal gravity gradient envelope;
                  the 3-sigma claim needs 2× envelope coverage [...]." .

The threshold values + evaluation method are buried in prose. An
audit can't:

  • Enumerate which threshold each sufficiency claim verified.
  • Show coverage matrices (which slew amplitudes were tested? which
    disturbance magnitudes? which evaluation methods?) across a fleet
    of similar requirements.
  • Statistically aggregate "of N requirements with 3-sigma claims,
    how many had Monte Carlo sufficiency evidence vs deterministic-
    worst-case?"

Why this matters

The whole point of the Hawkins-Habli ACP split (which v0.1 already
honors at the class level via separate Adequacy / Sufficiency
attestation classes) is to make these two epistemic moves
independently auditable. A model can be adequate (faithful to the
physics under its simplifications) but the evidence about it can be
insufficient (deterministic worst-case isn't statistical 3-sigma).
The class split captures this; the within-class content doesn't.

Without structural vocabulary inside each class:

  • The audit relies on a human reading the comment text to figure out
    the assumption/threshold structure.
  • The deprecation flow (when an assumption later turns out to be
    wrong) can't fire automatically — it can only fire when a human
    notices.
  • Multi-attestation aggregation policy (item What does sysml mean here #5 on the ADCS
    prototype's future work list) is gated on this; without per-
    assumption / per-threshold structure, "do N independent engineers
    agree on X?" can't be expressed.

Possible vocabulary additions

(Not exclusive; these compose.)

A — Reified assumptions + thresholds

<urn:rtm:attest/adequacy-ev-proof-req-001>
    a rtm:AdequacyAttestation ;
    rtm:reliesOnAssumption [
        a rtm:SimplifyingAssumption ;
        rdfs:label "rigid-body, no flex modes" ;
        rdfs:comment "Higher-order flex modes shown to be outside
                      controller bandwidth in <iri>." ;
    ] ,
    [
        a rtm:SimplifyingAssumption ;
        rdfs:label "gravity-gradient-only disturbance environment" ;
        rdfs:comment "Permanently out of scope for this requirement:
                      solar pressure, magnetic, flex, fuel slosh." ;
    ] .

<urn:rtm:attest/sufficiency-ev-sim-req-001>
    a rtm:SufficiencyAttestation ;
    rtm:verifiedThreshold [
        a rtm:ThresholdCriterion ;
        rtm:thresholdName "settling time" ;
        rtm:thresholdValue "120"^^xsd:double ;
        rtm:thresholdUnit "s" ;
        rtm:evaluationMethod rtm:deterministic-worst-case ;
    ] .

Pros: full SPARQL queryability; SHACL profiles can demand specific
assumptions/thresholds for certain classes of claims.

Cons: triple count grows linearly with audit depth; parsimony
budget pressure.

B — Subproperties of rdfs:comment

<urn:rtm:attest/adequacy-ev-proof-req-001>
    rtm:adequacyAssumption "rigid-body, no flex modes" ;
    rtm:adequacyAssumption "gravity-gradient-only disturbance" .

<urn:rtm:attest/sufficiency-ev-sim-req-001>
    rtm:sufficiencyCriterion "settling time ≤ 120s, deterministic" ;
    rtm:sufficiencyCriterion "slew amplitude tested: 10°" .

Pros: flatter; one predicate per fact; queryable.

Cons: still string-typed; no machine-readable thresholds.

C — OntoGSN promotion (paired with #29)

gsn:Assumption for adequacy assumptions; gsn:Justification for
sufficiency criteria; gsn:inContextOf for cross-attestation
dependencies. The MVC pattern doc names OntoGSN as one of the
integration ontologies; this would be its proper introduction.

Pros: standards-aligned (OntoGSN is a published ontology); pairs
with the broader GSN integration story.

Cons: parsimony budget cost; adopters need to learn GSN.

Coupling to #29

#29 asked WHERE engineering preconditions (operational envelopes,
mission phases) attach to in the graph. This issue asks HOW the
inside of an adequacy or sufficiency attestation is STRUCTURED to
capture the partition.

The two decisions are coupled — Option C above (GSN) would resolve
both simultaneously. Options A and B leave #29 separately undecided.

Impl-side follower

Once the vocab direction is picked, the engineer skill's attest
catechism gains class-specific follow-up questions:

  • For adequacy: "What simplifying assumptions does this claim rely
    on? (one per line)"
  • For sufficiency: "Which thresholds does this claim verify? Value
    • unit + evaluation method per threshold?"

The constructor's CLI grows matching flags
(--assumption='...' repeatable; `--threshold-name + --threshold-value

  • --threshold-unit + --evaluation-method` per threshold). Impl-side
    ticket lands once the spec direction is picked.

What the walkthrough did with REQ-001

For the UAT walkthrough we collapsed the partition into rdfs:comment
text per current v0.1 vocabulary. That works for the demo and
correctly surfaces the prototype's deliberately-failed REQ-001 case.
But the partition is recognized — every adequacy/sufficiency
attestation in the walkthrough now carries explicit per-class
language pending the structural vocab landing.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions