Skip to content

Improve secrets control gap classification#1612

Open
modelsbridgeaicom-ship-it wants to merge 1 commit into
UnitOneAI:mainfrom
modelsbridgeaicom-ship-it:improve/secrets-control-gap-classification
Open

Improve secrets control gap classification#1612
modelsbridgeaicom-ship-it wants to merge 1 commit into
UnitOneAI:mainfrom
modelsbridgeaicom-ship-it:improve/secrets-control-gap-classification

Conversation

@modelsbridgeaicom-ship-it

Copy link
Copy Markdown

Skill Improvement ($50-150 Bounty)

Skill Modified

Skill name: secrets-management
Skill path: skills/devsecops/secrets-management/

Addresses #1611.

What Was Wrong

The skill said missing secret detection tooling should not be counted as a secrets finding, but later classified "No secret detection tooling deployed" as Critical. That could make a clean repository with no exposed credential look like it has a Critical leaked-secret finding.

The skill also only considered repo-visible scanner config, so platform-native controls like GitHub Secret Protection or GitLab Secret Detection could be missed when no .gitleaks.toml, .trufflehog.yml, or .secrets.baseline is committed.

What This PR Fixes

  • Separates confirmed Secret Exposure findings from Secrets Control Gaps.
  • Removes scanner absence from Critical exposure classification unless an actual active or unrotated secret is present.
  • Adds platform-native scanner evidence for GitHub Secret Protection and GitLab Secret Detection.
  • Adds scanner surface tracking for repository files, git history, PRs, issues, discussions, wiki, release assets, and secret gists.
  • Adds validity and remediation states: active, inactive, revoked, rotated, unknown, provider-notified, and purged-from-history.
  • Updates output format with a dedicated Secrets Control Gaps table.
  • Adds two calibration fixtures showing the benign platform-scanner case and the vulnerable active-secret-plus-control-gap case.

Evidence

Before, this benign case could be overclassified:

No local scanner config is committed, but GitHub Secret Protection and push protection are enabled at the organization level and there are no active scanner alerts.

After, this is recorded as platform-native scanner coverage and does not create a Critical leaked-secret finding.

Before, this real exposure could be mixed with tooling gaps:

An active production deploy token is present in configuration, and scanner/history controls are missing.

After, this creates one Critical Secret Exposure finding plus a separate High Secrets Control Gap. The control gap supports remediation priority but is not counted as a second leaked-secret finding.

Test Cases Added/Updated

  • Added benign fixture: skills/devsecops/secrets-management/tests/benign/platform-native-scanning-no-local-config.md
  • Added vulnerable fixture: skills/devsecops/secrets-management/tests/vulnerable/active-secret-with-control-gap.md
  • Existing frontmatter validation still passes

Validation

  • git diff --check
  • Full frontmatter required-field check over skills/ and roles/
  • Changed-file prompt-injection pattern scan
  • Marker checks for Secrets Control Gaps, GitHub Secret Protection, GitLab Secret Detection, Validity State, Remediation State, Not Evaluable, and platform-native scanner wording
  • Fixture safety check to avoid raw secret-like token patterns in added fixtures

Bounty Tier

  • Minor ($50) - Doc update, small logic tweak, typo fix
  • Moderate ($100) - False-positive reduction with evidence and output-format changes
  • Substantial ($150) - Rewritten detection logic, major coverage expansion

Bounty Info

  • I have read and agree to the CONTRIBUTING.md bounty terms
  • Preferred payment method: PayPal or crypto; details can be provided privately after maintainer acceptance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant