Skip to content

Commit 73b4f8f

Browse files
dcramerclaude
andauthored
feat(ci): Add logging-observability skill for Warden (#766)
- Adds new `logging-observability` skill that reviews code for correct logging and error handling patterns - Enforces the rule that 4xx errors never create Sentry issues while 5xx errors always do - Adds path filters (`**/*.ts`, `**/*.tsx`) to restrict both skills to TypeScript files only Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
1 parent 97848cc commit 73b4f8f

2 files changed

Lines changed: 315 additions & 0 deletions

File tree

Lines changed: 303 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,303 @@
1+
---
2+
name: logging-observability
3+
description: Review code for correct logging and error handling patterns. Use when reviewing code that handles errors, uses logging functions, or captures exceptions. Enforces the error hierarchy where 4xx errors are never logged to Sentry and 5xx errors always are. Trigger phrases include "review logging", "check error handling", "audit observability", or verify correct use of logIssue vs logError.
4+
---
5+
6+
# Logging & Observability Correctness
7+
8+
Enforce correct logging and error handling patterns in the sentry-mcp codebase. The core principle: **4xx errors (user-correctable) must NEVER create Sentry issues. 5xx errors (system failures) must ALWAYS create Sentry issues.**
9+
10+
## Error Hierarchy
11+
12+
### API Errors (`packages/mcp-core/src/api-client/errors.ts`)
13+
14+
```
15+
ApiError (base class)
16+
├─ ApiClientError (4xx - user errors, NOT sent to Sentry)
17+
│ ├─ ApiPermissionError (403)
18+
│ ├─ ApiNotFoundError (404)
19+
│ ├─ ApiValidationError (400, 422)
20+
│ ├─ ApiAuthenticationError (401)
21+
│ └─ ApiRateLimitError (429)
22+
└─ ApiServerError (5xx - system errors, SENT to Sentry)
23+
```
24+
25+
**4xx errors (`ApiClientError`)**: User input errors. The user can fix these by correcting their request. These must NEVER create Sentry issues.
26+
27+
**5xx errors (`ApiServerError`)**: System failures. These indicate server-side problems outside user control. These must ALWAYS create Sentry issues.
28+
29+
### Application Errors (`packages/mcp-core/src/internal/errors.ts`)
30+
31+
| Error Type | Sentry Issue? | Use Case |
32+
|------------|---------------|----------|
33+
| `UserInputError` | No | Validation failures the user can fix |
34+
| `ConfigurationError` | No | Missing or invalid configuration |
35+
| `LLMProviderError` | No | AI provider service unavailable (e.g., region restrictions) |
36+
37+
## Logging Functions (`packages/mcp-core/src/telem/logging.ts`)
38+
39+
| Function | Creates Sentry Issue? | Use For |
40+
|----------|----------------------|---------|
41+
| `logDebug()` | No | Development debugging information |
42+
| `logInfo()` | No | Routine operational information |
43+
| `logWarn()` | No | Expected errors (4xx), recoverable conditions |
44+
| `logError()` | No | Errors that do NOT need alerting |
45+
| `logIssue()` | **Yes** | System failures (5xx), unexpected errors that need investigation |
46+
47+
### When to Use Each Function
48+
49+
**`logDebug()`**: Use for verbose debugging output that helps trace execution flow. Only visible when LOG_LEVEL=debug.
50+
51+
**`logInfo()`**: Use for routine operational events - tool calls, API requests, cache hits. Normal system behavior.
52+
53+
**`logWarn()`**: Use for expected error conditions - API 4xx responses, validation failures, rate limits. The system handled these gracefully.
54+
55+
**`logError()`**: Use for error conditions that do NOT need a Sentry issue. This logs at error level but does NOT create an issue.
56+
57+
**`logIssue()`**: Use ONLY for errors requiring investigation - 5xx responses, unexpected exceptions, system failures. This creates a Sentry issue with an Event ID.
58+
59+
## Anti-Patterns to Catch
60+
61+
### 1. Logging 4xx Errors with `logIssue()`
62+
63+
```typescript
64+
// WRONG: Creates Sentry noise from user errors
65+
if (error instanceof ApiClientError) {
66+
logIssue(error); // Creates unnecessary Sentry issue
67+
}
68+
69+
// CORRECT: Log without creating an issue
70+
if (error instanceof ApiClientError) {
71+
logWarn(error, { loggerScope: ["api", "client-error"] });
72+
}
73+
```
74+
75+
**Why this matters**: 4xx errors are user-correctable. Creating Sentry issues for them generates noise that drowns out real system failures.
76+
77+
### 2. Missing `logIssue()` for 5xx Errors
78+
79+
```typescript
80+
// WRONG: Server errors go undetected
81+
if (error instanceof ApiServerError) {
82+
logError(error); // No Sentry issue created
83+
return formatError(error);
84+
}
85+
86+
// CORRECT: Create Sentry issue for system failures
87+
if (error instanceof ApiServerError) {
88+
const eventId = logIssue(error);
89+
return formatErrorWithEventId(error, eventId);
90+
}
91+
```
92+
93+
**Why this matters**: 5xx errors indicate system failures. Without Sentry issues, these go undetected and unresolved.
94+
95+
### 3. Using Raw `console.log/warn/error`
96+
97+
```typescript
98+
// WRONG: Bypasses structured logging
99+
console.error("Something went wrong:", error);
100+
101+
// CORRECT: Use structured logging
102+
logError(error, {
103+
loggerScope: ["my-tool"],
104+
extra: { context: "additional info" },
105+
});
106+
```
107+
108+
**Why this matters**: Raw console methods bypass Sentry integration, lose structured context, and make debugging harder.
109+
110+
### 4. Catching All Errors Indiscriminately
111+
112+
```typescript
113+
// WRONG: Treats all errors the same
114+
try {
115+
await apiService.fetchData();
116+
} catch (error) {
117+
logIssue(error); // Even 404s create issues
118+
}
119+
120+
// CORRECT: Discriminate by error type
121+
try {
122+
await apiService.fetchData();
123+
} catch (error) {
124+
if (error instanceof ApiServerError) {
125+
logIssue(error);
126+
} else if (error instanceof ApiClientError) {
127+
logWarn(error, { loggerScope: ["api"] });
128+
} else {
129+
logIssue(error); // Unknown errors need investigation
130+
}
131+
}
132+
```
133+
134+
### 5. Missing Structured Context
135+
136+
```typescript
137+
// WRONG: No context for debugging
138+
logIssue(error);
139+
140+
// CORRECT: Include structured context
141+
logIssue(error, {
142+
loggerScope: ["tool-name", "operation"],
143+
contexts: {
144+
request: { organizationSlug, projectSlug },
145+
},
146+
extra: { attemptCount: 3 },
147+
});
148+
```
149+
150+
**Why this matters**: Structured context (`loggerScope`, `extra`, `contexts`) enables filtering and searching in Sentry and log aggregation systems.
151+
152+
### 6. Using `captureException()` Directly
153+
154+
```typescript
155+
// WRONG: Bypasses logging wrapper
156+
import { captureException } from "@sentry/core";
157+
captureException(error);
158+
159+
// CORRECT: Use the logging wrapper
160+
logIssue(error, { contexts: { ... } });
161+
```
162+
163+
**Why this matters**: `logIssue()` provides consistent formatting, adds structured context, and logs to both console and Sentry.
164+
165+
## Correct Patterns
166+
167+
### Tool Handler Pattern
168+
169+
Let errors bubble up to the MCP server wrapper. Do NOT catch errors unless adding value.
170+
171+
```typescript
172+
// CORRECT: Let errors bubble naturally
173+
export async function handleTool(params: Params) {
174+
// API client throws typed errors via createApiError factory
175+
const result = await apiService.someMethod(params);
176+
return formatResult(result);
177+
// Errors bubble to MCP server -> formatErrorForUser handles them
178+
}
179+
```
180+
181+
The MCP server wrapper (`formatErrorForUser`) automatically:
182+
- Formats `ApiClientError` as "Input Error" (no Sentry issue)
183+
- Formats `ApiServerError` as "Error" with Event ID (creates Sentry issue)
184+
- Formats `UserInputError` as "Input Error" (no Sentry issue)
185+
186+
### Agent Tool Pattern
187+
188+
Use `agentTool()` wrapper for embedded agent tools. It handles all error cases.
189+
190+
```typescript
191+
import { agentTool } from "../../internal/agents/tools/utils";
192+
193+
export function createMyTool(apiService: SentryApiService) {
194+
return agentTool({
195+
description: "Tool description",
196+
parameters: z.object({ param: z.string() }),
197+
execute: async (params) => {
198+
// No error handling needed - agentTool handles it
199+
const data = await apiService.someMethod(params);
200+
return formatResult(data);
201+
},
202+
});
203+
}
204+
```
205+
206+
The `agentTool()` wrapper automatically:
207+
- Returns `{ result: data }` on success
208+
- Returns `{ error: "Input Error: ..." }` for `UserInputError` and `ApiClientError`
209+
- Returns `{ error: "Server Error: ..." }` with Event ID for `ApiServerError`
210+
- Uses `logWarn()` for 4xx errors (no Sentry issue)
211+
- Uses `logIssue()` for 5xx errors (creates Sentry issue)
212+
213+
### Graceful Degradation with Error Discrimination
214+
215+
When implementing fallback behavior, discriminate errors properly.
216+
217+
```typescript
218+
async function fetchWithFallback() {
219+
try {
220+
return await primarySource.fetch();
221+
} catch (error) {
222+
if (error instanceof ApiClientError) {
223+
// 4xx: User error, don't retry or fall back
224+
throw error;
225+
}
226+
if (error instanceof ApiServerError) {
227+
// 5xx: System error, log and try fallback
228+
logIssue(error, {
229+
loggerScope: ["fetch", "primary-failed"],
230+
extra: { fallbackAttempted: true },
231+
});
232+
return await fallbackSource.fetch();
233+
}
234+
// Unknown error: Log and rethrow
235+
logIssue(error, { loggerScope: ["fetch", "unexpected"] });
236+
throw error;
237+
}
238+
}
239+
```
240+
241+
### Including Event ID in User-Facing Errors
242+
243+
When system errors reach users, include the Event ID for support reference.
244+
245+
```typescript
246+
if (error instanceof ApiServerError) {
247+
const eventId = logIssue(error, {
248+
contexts: { request: { url, method } },
249+
});
250+
return {
251+
error: `Server error. Event ID: ${eventId}. Contact support with this ID.`,
252+
};
253+
}
254+
```
255+
256+
## Review Checklist
257+
258+
When reviewing code for logging correctness, verify each item:
259+
260+
### Error Classification
261+
262+
- [ ] `ApiClientError` (4xx) uses `logWarn()`, NOT `logIssue()`
263+
- [ ] `ApiServerError` (5xx) uses `logIssue()`, NOT `logWarn()` or `logError()`
264+
- [ ] `UserInputError` uses `logWarn()`, NOT `logIssue()`
265+
- [ ] Unknown/unexpected errors use `logIssue()` for investigation
266+
267+
### Logging Function Usage
268+
269+
- [ ] No raw `console.log/warn/error` calls (use `logDebug/Info/Warn/Error/Issue`)
270+
- [ ] No direct `captureException()` calls (use `logIssue()` wrapper)
271+
- [ ] Appropriate log level for the situation (debug vs info vs warn vs error)
272+
273+
### Structured Context
274+
275+
- [ ] `loggerScope` identifies the component (e.g., `["tool-name", "operation"]`)
276+
- [ ] `contexts` includes request-level info (org, project, resource IDs)
277+
- [ ] `extra` includes debugging details (attempt counts, durations, flags)
278+
279+
### Error Handling Patterns
280+
281+
- [ ] Tool handlers let errors bubble (no unnecessary try/catch)
282+
- [ ] Agent tools use `agentTool()` wrapper
283+
- [ ] Error discrimination happens before logging decisions
284+
- [ ] Event IDs returned to users for 5xx errors
285+
286+
### Common Issues
287+
288+
| Pattern Found | Severity | Action |
289+
|---------------|----------|--------|
290+
| `logIssue(ApiClientError)` | High | Change to `logWarn()` |
291+
| Missing `logIssue(ApiServerError)` | High | Add `logIssue()` call |
292+
| Raw `console.error()` | Medium | Change to `logError()` or `logIssue()` |
293+
| `captureException()` directly | Medium | Change to `logIssue()` |
294+
| Missing `loggerScope` | Low | Add component identifier |
295+
| Missing `contexts`/`extra` | Low | Add structured debugging info |
296+
297+
## Reference Files
298+
299+
- Logging functions: `packages/mcp-core/src/telem/logging.ts`
300+
- Error hierarchy: `packages/mcp-core/src/api-client/errors.ts`
301+
- Error formatting: `packages/mcp-core/src/internal/error-handling.ts`
302+
- Agent tool wrapper: `packages/mcp-core/src/internal/agents/tools/utils.ts`
303+
- Full error handling guide: `docs/error-handling.md`

‎warden.toml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,3 +5,15 @@ name = "testing-guidelines"
55
event = "pull_request"
66
actions = ["opened", "synchronize", "reopened"]
77
skill = "testing-guidelines"
8+
9+
[triggers.filters]
10+
paths = ["**/*.ts", "**/*.tsx"]
11+
12+
[[triggers]]
13+
name = "logging-observability"
14+
event = "pull_request"
15+
actions = ["opened", "synchronize", "reopened"]
16+
skill = "logging-observability"
17+
18+
[triggers.filters]
19+
paths = ["**/*.ts", "**/*.tsx"]

0 commit comments

Comments
 (0)