Skip to content

Commit f11d7f6

Browse files
docs(red-team): follow-up fixes to Quick Start (#325)
- Step 2: point users to app.langwatch.ai Settings → API Keys so the LANGWATCH_API_KEY placeholder isn't a dead end for new users. - Step 4: replace hardcoded pytest/npm commands with "run your tests or ask your coding assistant to run and monitor it". The hardcoded paths assumed a directory structure the MCP may not produce, and npm test/pytest defaults don't give red team runs enough timeout. Also surface cost/time awareness (50 turns × LLM calls on two models = real tokens). - Step 5: drop "(optional)" — viewing runs in LangWatch is the natural next step, not an aside. Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
1 parent 62a22f7 commit f11d7f6

1 file changed

Lines changed: 3 additions & 13 deletions

File tree

docs/docs/pages/advanced/red-teaming/quick-start.mdx

Lines changed: 3 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ Works with Claude Code, Cursor, Claude Desktop, Codex, and any MCP-compatible cl
4747
}
4848
```
4949

50-
Restart the client. Full reference: [LangWatch MCP Server](https://langwatch.ai/docs/integration/mcp).
50+
Grab your API key from [app.langwatch.ai](https://app.langwatch.ai)**Settings → API Keys**, then restart your client. Full reference: [LangWatch MCP Server](https://langwatch.ai/docs/integration/mcp).
5151

5252
## 3. Ask your assistant to generate the test
5353

@@ -150,21 +150,11 @@ describe("Agent security", () => {
150150

151151
## 4. Run it
152152

153-
:::code-group
154-
155-
```bash [python]
156-
pytest tests/red_team/ -v
157-
```
158-
159-
```bash [typescript]
160-
npm test -- tests/red-team
161-
```
162-
163-
:::
153+
Run your usual test command, or just ask your coding assistant to run it and monitor it for you. Red team runs are long — 50 turns can take several minutes and will consume real LLM tokens on both the attacker and target models — so make sure your runner's per-test timeout is generous.
164154

165155
Each turn prints the attacker's message, your agent's response, and a per-turn score. A failing test includes the full transcript and the judge's reasoning — you see exactly which turn broke the agent and how.
166156

167-
## 5. View the run in LangWatch (optional)
157+
## 5. View the run in LangWatch
168158

169159
If you've instrumented your agent with LangWatch, every red team run appears in the Simulations dashboard: full attack transcripts, per-turn scores, and side-by-side comparison across runs to track whether a prompt change made your agent more or less resilient.
170160

0 commit comments

Comments
 (0)