Skip to content

fix(redteam): derive attack successful from threat field#219

Merged
cdot65 merged 1 commit into
mainfrom
cdot65/fix/redteam-attacks-bypassed-from-threat
May 28, 2026
Merged

fix(redteam): derive attack successful from threat field#219
cdot65 merged 1 commit into
mainfrom
cdot65/fix/redteam-attacks-bypassed-from-threat

Conversation

@cdot65

@cdot65 cdot65 commented May 28, 2026

Copy link
Copy Markdown
Owner

Summary

  • airs redteam report <jobId> --attacks printed BLOCKED for every attack regardless of outcome.
  • Root cause: listAttacks normalizer at src/airs/redteam.ts:467 lifted a.successful — a field the API doesn't return per airs --debug capture. Result was always undefined (falsy) → renderer always picked BLOCKED.
  • Fix: derive successful from a.threat in the normalizer. Renderer logic unchanged (was already correct given a correct successful value).
  • Updated existing listAttacks test mocks to reflect real API shape (threat instead of stale successful).

Test plan

  • RED: existing listAttacks test now fails (mocks switched to threat); new test confirms missing threatsuccessful: false; renderer regression tests for BYPASSED/BLOCKED
  • GREEN: 685/685 tests pass
  • pnpm lint clean
  • pnpm tsc --noEmit clean
  • pnpm test 685 passed
  • pnpm docs:check clean

Closes #205

The listAttacks normalizer was reading `a.successful` — a field the API
doesn't return. As a result every attack rendered as `BLOCKED` even when
`threat: true` (bypassed). Derive successful from `a.threat` so the
report --attacks pretty output labels outcomes correctly.

Closes #205
@cdot65 cdot65 merged commit 03ea286 into main May 28, 2026
5 checks passed
@cdot65 cdot65 deleted the cdot65/fix/redteam-attacks-bypassed-from-threat branch May 28, 2026 02:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

redteam report --attacks: mis-labels bypassed attacks as BLOCKED

1 participant