Update docs: standard presets, MCP mock mode, --test-connection, plugin install

mnvsk97 · claude · mnvsk97 · commit fc9c2bd075f1 · 2026-04-04T21:28:32.000-07:00
- CLI ref: add --test-connection flag, update init description
- Scenarios ref: add standard/standard-mcp/standard-all presets
- Plugin page: full install commands, init flow with mode selection
- Quickstart: MCP mock mode, validate --test-connection

Co-Authored-By: Claude Opus 4.6 (1M context) &lt;noreply@anthropic.com&gt;
diff --git a/docs-site/docs/getting-started/plugin.md b/docs-site/docs/getting-started/plugin.md
@@ -4,35 +4,54 @@ AgentBreak works as a plugin for Claude Code, giving Claude structured commands
 
 ## Install
 
+1. Install the package:
+
 ```bash
 pip install agentbreak
 ```
 
-## Configure
-
-In Claude Code:
+2. Add the plugin in Claude Code:
 
 ```
 /plugin marketplace add mnvsk97/agentbreak
 /plugin install agentbreak@mnvsk97-agentbreak
 /reload-plugins
 ```
 
+To update after a new release:
+
+```
+/plugin marketplace add mnvsk97/agentbreak
+/reload-plugins
+```
+
 ## Commands
 
 | Command | What it does |
 |---------|-------------|
 | `/agentbreak:init` | Initialize AgentBreak, analyze your agent codebase |
-| `/agentbreak:create-tests` | Generate tailored chaos scenarios |
+| `/agentbreak:create-tests` | Generate project-specific chaos scenarios |
 | `/agentbreak:run-tests` | Run tests and produce a resilience report |
 
 ### `/agentbreak:init`
 
-Sets up `.agentbreak/` config and scans your codebase for provider, framework, MCP tools, and error handling patterns. Run this first.
+Full setup flow:
+
+1. Checks AgentBreak is installed
+2. Runs `agentbreak init` to create `.agentbreak/`
+3. Analyzes your codebase (provider, framework, MCP tools, error handling)
+4. Asks: **mock or proxy mode?**
+    - **Mock** — no API keys needed, synthetic responses
+    - **Proxy** — real API traffic, requires valid keys
+5. Writes `application.yaml` and `scenarios.yaml` (with standard preset)
+6. Validates config (+ `--test-connection` if proxy mode)
+7. Offers to generate project-specific scenarios
 
 ### `/agentbreak:create-tests`
 
-Generate `scenarios.yaml` entries based on the analysis from init. Claude understands the full scenario schema (8 fault kinds, 3 schedule modes, match filters) and writes scenarios targeting your agent's specific failure modes.
+Generates project-specific scenarios on top of the standard preset. Analyzes your codebase to find specific MCP tools, models, and integrations, then writes targeted fault scenarios. Can be run anytime to add more scenarios.
+
+Standard baseline scenarios are always included via the preset — this command focuses on what's unique to your agent.
 
 ### `/agentbreak:run-tests`
 
diff --git a/docs-site/docs/getting-started/quickstart.md b/docs-site/docs/getting-started/quickstart.md
@@ -16,8 +16,8 @@ agentbreak init
 
 This creates `.agentbreak/` with two config files:
 
-- **`application.yaml`** — what to proxy (LLM mode, MCP upstream, port)
-- **`scenarios.yaml`** — what faults to inject
+- **`application.yaml`** — what to proxy (LLM/MCP mode, upstream URLs, port)
+- **`scenarios.yaml`** — what faults to inject (includes a standard preset with baseline scenarios)
 
 ## 3. Start the proxy
 
@@ -62,7 +62,9 @@ You'll get a JSON response with:
 
 ## Mock vs proxy mode
 
-By default, AgentBreak runs in **mock mode** — no API key needed, it returns synthetic responses. To test against the real API:
+By default, AgentBreak runs in **mock mode** — no API keys needed, it returns synthetic responses. Both LLM and MCP support mock mode.
+
+To test against real APIs, switch to **proxy mode**:
 
 ```yaml
 # .agentbreak/application.yaml
@@ -72,7 +74,18 @@ llm:
   upstream_url: https://api.openai.com    # or https://api.anthropic.com
   auth:
     type: bearer
-    token_env: OPENAI_API_KEY             # or ANTHROPIC_API_KEY
+    env: OPENAI_API_KEY                   # or ANTHROPIC_API_KEY
+
+mcp:
+  enabled: true
+  mode: proxy
+  upstream_url: http://localhost:8001/mcp
+```
+
+Verify your credentials before running tests:
+
+```bash
+agentbreak validate --test-connection
 ```
 
 ## Next steps
diff --git a/docs-site/docs/reference/cli.md b/docs-site/docs/reference/cli.md
@@ -4,7 +4,7 @@ All commands work from your project root (where `.agentbreak/` lives).
 
 ## `agentbreak init`
 
-Create `.agentbreak/` directory with default `application.yaml` and `scenarios.yaml`.
+Create `.agentbreak/` directory with default `application.yaml` and `scenarios.yaml`. Auto-detects your provider (OpenAI/Anthropic), framework, and MCP usage. Generates `scenarios.yaml` with a standard preset based on what's enabled.
 
 ```bash
 agentbreak init
@@ -26,8 +26,17 @@ Check your config files for errors.
 
 ```bash
 agentbreak validate
+agentbreak validate --test-connection   # also test upstream auth (proxy mode)
 ```
 
+The `--test-connection` flag makes a lightweight request to each proxy-mode upstream to verify connectivity and auth. Results:
+
+- **OK** — upstream reachable, auth works
+- **AUTH FAILED (401)** — bad API key or token
+- **FORBIDDEN (403)** — key valid but insufficient permissions
+- **CONNECTION FAILED** — wrong URL or upstream is down
+- **TIMEOUT** — upstream too slow to respond
+
 ## `agentbreak inspect`
 
 Connect to your MCP server and discover available tools, resources, and prompts. Writes `.agentbreak/registry.json`.
diff --git a/docs-site/docs/reference/scenarios.md b/docs-site/docs/reference/scenarios.md
@@ -86,31 +86,57 @@ scenarios:
 
 ## Presets
 
-Skip manual config with built-in bundles:
+Use presets for baseline coverage out of the box:
 
 ```yaml
-preset: brownout
+version: 1
+preset: standard
 ```
 
-Combine a preset with custom scenarios:
+Combine a preset with project-specific scenarios:
 
 ```yaml
-preset: brownout
+version: 1
+preset: standard-all
 scenarios:
-  - name: custom-fault
-    summary: My extra fault
+  - name: search-tool-timeout
+    summary: search_docs times out
     target: mcp_tool
+    match:
+      tool_name: search_docs
     fault:
-      kind: http_error
-      status_code: 503
+      kind: timeout
+      min_ms: 5000
+      max_ms: 10000
     schedule:
       mode: random
       probability: 0.2
 ```
 
-| Preset | What it does |
-|--------|-------------|
-| `brownout` | Random LLM latency + rate limits |
-| `mcp-slow-tools` | 90% of MCP tool calls are slow |
-| `mcp-tool-failures` | 30% of MCP tool calls return 503 |
-| `mcp-mixed-transient` | Light MCP latency + errors |
+Use multiple presets:
+
+```yaml
+version: 1
+presets:
+  - standard
+  - mcp-slow-tools
+```
+
+### Standard presets
+
+These provide baseline coverage. `agentbreak init` uses these by default.
+
+| Preset | Target | Scenarios |
+|--------|--------|-----------|
+| `standard` | LLM | 6 scenarios: rate limit (429), server error (500), latency (3-8s), invalid JSON, empty response, schema violation |
+| `standard-mcp` | MCP | 7 scenarios: 503 unavailable, timeout (5-15s), latency (3-8s), empty response, invalid JSON, schema violation, wrong content |
+| `standard-all` | Both | All 13 baseline scenarios (standard + standard-mcp) |
+
+### Specialty presets
+
+| Preset | Target | What it does |
+|--------|--------|-------------|
+| `brownout` | LLM | Random LLM latency + rate limits |
+| `mcp-slow-tools` | MCP | 90% of MCP tool calls are slow |
+| `mcp-tool-failures` | MCP | 30% of MCP tool calls return 503 |
+| `mcp-mixed-transient` | MCP | Light MCP latency + errors |
