feat: add auth and resilience scenarios

[daranium2020]

Summary

Implements v0.3.0 config-driven scenario runner with RFC 6750-compliant auth classification and explicit retry categories.

Auth classification

Finding code	Condition
AUTH_REQUIRED	401 with no credentials supplied
AUTH_INVALID	401 with credentials + any rejection, including error="invalid_token" (covers expired, revoked, and malformed per RFC 6750 §3.1)
AUTH_EXPIRED	401 with credentials + unambiguous expiry indicator only: error="expired" or error="token_expired"
AUTH_FORBIDDEN	403 regardless of credentials

The key correctness fix: error="invalid_token" now produces AUTH_INVALID, not AUTH_EXPIRED. RFC 6750 §3.1 explicitly defines that code as covering expired, revoked, AND malformed tokens — it is too broad to classify as expiry-specific.

Retry semantics (explicit opt-in, off by default)

Condition	retryOn category	Default
HTTP 429 (RATE_LIMITED)	rate-limit	off
5xx responses (REMOTE_HTTP_ERROR)	server-error	off
Connection failures and connect timeouts	connection-failure	off
Response timeouts	response-timeout	off
401, 403	—	Never retried
400, schema errors, malformed MCP	—	Never retried
Scenario timeout	—	Never retried

A retry fires only when maxAttempts > 1 AND the failure category is listed in retryOn.

Config file support (mcp-release.config.yml)

version: 1
endpoint: https://api.example.com/mcp
headers:
  Authorization: Bearer ${API_TOKEN}
timeouts:
  connectMs: 5000
  responseMs: 10000
retries:
  maxAttempts: 3
  backoffMs: 500
  retryOn:
    - rate-limit
    - server-error
scenarios:
  - name: healthy
    expect:
      result: pass
  - name: missing-auth
    removeHeaders:
      - Authorization
    expect:
      httpStatus: 401

• ${VAR_NAME} placeholders resolved from environment variables at runtime
• Per-scenario header overrides and removeHeaders
• Structured retries block with explicit opt-in categories

Reporting

• Terminal: shows 2/3 attempts, rate-limit retry on retried scenarios
• Markdown: adds Retry column to scenario summary table
• JSON: round-trips maxAttempts and retryCategory fields
• GitHub Actions: sets step outputs for config-mode runs

Other additions

• HTTP 429 Retry-After parsing (integer seconds and HTTP-date)
• RATE_LIMITED finding code with remediation guidance
• Timeout classification: CONNECT_TIMEOUT vs RESPONSE_TIMEOUT
• Credential redaction in all report formats
• 883 tests pass across 41 test files

Test plan

• pnpm typecheck — clean
• pnpm lint — clean
• pnpm test — 883/883 pass (41 test files)
• pnpm build — clean, all packages including apps/web
• No conflict markers in source files
• All package versions remain at 0.2.1 (no version bump)
• No .env.local or credentials exposed
• Web app SSRF protections unchanged

Files changed

New files (13): fixtures/servers/src/auth-scenarios.ts, packages/cli/src/config.ts, packages/cli/tests/config.test.ts, packages/core/src/config-report.ts, packages/core/src/rate-limit.ts, packages/core/src/scenario-runner.ts, packages/core/tests/auth-invalid.test.ts, packages/core/tests/rate-limit.test.ts, packages/core/tests/scenario-runner.test.ts, packages/reporter/src/config-json.ts, packages/reporter/src/config-markdown.ts, packages/reporter/src/config-terminal.ts, packages/reporter/tests/config-reporter.test.ts

Modified files (22): README.md, apps/web/src/app/docs/page.tsx, apps/web/tests/ui/DocsPage.test.tsx, and core/cli/reporter source and test files

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
mcp-release	Ready	Preview, Comment	Jul 19, 2026 11:29pm