Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions rfc-safe-proposal.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,8 @@ When doing so does not expose sensitive evidence or create additional risk, SAFE

Detection rules published to the catalog should be expressed in open, executable, vendor-neutral formats rather than prose, and should carry the same discipline the recommendations themselves require: declared true-positive and true-negative test cases, measured false-positive evidence on benign traffic, a version, and the technique classification above. Open rule standards with these properties already exist at both layers of the operating stack — for example Sigma for infrastructure telemetry, and ATR (Agent Threat Rules, MIT) for the agent layer of prompts, tool calls and skill artifacts, which already carries crosswalks to ATLAS and the OWASP classes. A catalog entry that names its technique, ships its tests and states its measured error rates is a control a member can adopt and verify; one that does not is advice.

Each published test result should identify the observed rule or ruleset digest; evaluator, configuration and normalization versions; corpus identity, digest and label provenance; observation window; evaluation unit, denominator and base rate; replay or live status; monitor, shadow or enforce mode; observed or blocked outcome; a correlation reference for the observation and decision path; and overlap or union semantics when multiple rules fire. Per-case outcomes should distinguish true positive, true negative, false positive, false negative, abstention, error and timeout, while separately reporting malformed, dropped, not-reached and unusable evidence. Without this envelope, two members can test the same named rule on the same named corpus while loading different artifacts, counting different units or silently treating failures as negatives, and publish incompatible error rates that appear equivalent.

For unintended access to real systems, recommendations might include default-deny network egress, explicit target allowlists, signed evaluation manifests, independent preflight isolation checks, real-time action monitoring, automatic stops when scope is uncertain and equivalent assurance requirements for evaluation partners.

# The Compact
Expand Down