Skip to content
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ The toolkit bundles the following capabilities as a single **mc-agent-toolkit**

| Feature | Description | Details |
|---|---|---|
| **Monitoring Advisor** | Analyzes data coverage across warehouses and use cases, identifies monitoring gaps, and creates monitors to protect critical data. | [README](skills/monitoring-advisor/README.md) |
| **Monitor Creation** | Guides AI agents through creating monitors correctly — validates tables, fields, and parameters before generating monitors-as-code YAML. | [README](skills/monitor-creation/README.md) |
| **Prevent** | Surfaces lineage, alerts, and blast radius before code changes. Generates monitors-as-code and targeted validation queries to prevent data incidents. | [README](skills/prevent/README.md) |
| **Generate Validation Notebook** | Generates SQL validation notebooks for dbt model changes, with targeted queries comparing baseline and development data. | [README](skills/generate-validation-notebook/README.md) |
Expand Down
1 change: 1 addition & 0 deletions plugins/claude-code/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
"prevent",
"push-ingestion",
"validation-notebook",
"monitoring-advisor",
"dbt",
"schema"
],
Expand Down
1 change: 1 addition & 0 deletions plugins/claude-code/skills/monitoring-advisor
1 change: 1 addition & 0 deletions skills/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Skills are platform-agnostic instruction sets that tell an AI coding agent what

| Skill | Description |
|---|---|
| **[Monitoring Advisor](monitoring-advisor/)** | Analyzes data coverage across warehouses and use cases, identifies monitoring gaps, and creates monitors to protect critical data. |
| **[Monitor Creation](monitor-creation/)** | Guides AI agents through creating monitors correctly — validates tables, fields, and parameters before generating monitors-as-code YAML. |
| **[Prevent](prevent/)** | Surfaces Monte Carlo context (lineage, alerts, blast radius) before code changes, generates monitors-as-code, and produces targeted validation queries. |
| **[Generate Validation Notebook](generate-validation-notebook/)** | Generates SQL validation notebooks for dbt model changes, with targeted queries comparing baseline and development data. |
Expand Down
17 changes: 13 additions & 4 deletions skills/monitor-creation/references/metric-monitor.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,13 @@ Use a metric monitor when the user wants to:
| `name` | string | Unique identifier for the monitor. Use a descriptive slug (e.g., `orders_null_check`). |
| `description` | string | Human-readable description of what the monitor checks. |
| `table` | string | Table MCON (preferred) or `database:schema.table` format. If not MCON, also pass `warehouse`. |
| `aggregate_time_field` | string | **MUST be a real timestamp/datetime column from the table.** NEVER guess this value. |
| `alert_conditions` | array | List of alert condition objects (see Alert Conditions below). |

## Optional Parameters

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `aggregate_time_field` | string | none | Timestamp/datetime column for time-windowed aggregation. **When provided, MUST be a real column from the table — NEVER guess this value.** When omitted, the monitor queries all rows on each run (whole-table scan). Omit for tables without a suitable timestamp column. |
| `warehouse` | string | auto-resolved | Warehouse name or UUID. Required if `table` is not an MCON. |
| `segment_fields` | array of string | none | Fields to group/segment metrics by (e.g., `["country", "status"]`). |
| `aggregate_by` | string | `"day"` | Time interval: `"hour"`, `"day"`, `"week"`, `"month"`. |
Expand All @@ -54,7 +54,16 @@ For example, to run a daily-aggregated monitor every other day, pass `aggregate_

## Choosing the Timestamp Field

The `aggregate_time_field` is the most critical parameter. It MUST be a real column from the table that contains timestamp or datetime values. This is the number one source of monitor creation failures.
The `aggregate_time_field` controls whether the monitor uses time-windowed aggregation or whole-table scans. When provided, it MUST be a real column from the table — this is the number one source of monitor creation failures.

### When to omit it

Omit `aggregate_time_field` when:
- The table has **no timestamp or datetime columns** at all.
- The table uses a **truncate-and-reload** pattern (fully replaced on each pipeline run) — time-windowed aggregation is meaningless since all rows share the same load time.
- The user wants to monitor the **entire table state** on each run (e.g., `RELATIVE_ROW_COUNT` segmented by a dimension).

When omitted, the monitor queries all rows on each run. This works well for small-to-medium tables but can be expensive for very large tables.

### How to pick it

Expand All @@ -63,9 +72,9 @@ The `aggregate_time_field` is the most critical parameter. It MUST be a real col
3. If the user specified one, verify it exists in the column list.
4. If exactly one obvious candidate exists, suggest it.
5. If multiple candidates exist, present them and ask the user.
6. If NO obvious timestamp columns exist, suggest a custom SQL monitor instead (which does not need a timestamp field).
6. If NO obvious timestamp columns exist, omit the field — the monitor will do a whole-table scan. For very large tables, consider whether a custom SQL monitor would be more efficient.

**NEVER** proceed without confirming the timestamp field exists in the table schema.
**NEVER** guess a timestamp field name — either confirm it exists in the schema or omit it.

### Common timestamp field mistakes

Expand Down
83 changes: 83 additions & 0 deletions skills/monitoring-advisor/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Monte Carlo Monitoring Advisor Skill

Analyze data coverage across warehouses and use cases, identify monitoring gaps, and create monitors to protect critical data. Walks users through warehouse discovery, use-case exploration, coverage gap analysis, and monitor creation — all through natural conversation.

## Editor & Stack Compatibility

The skill works with any AI editor that supports MCP and the Agent Skills format — including Claude Code, Cursor, and VS Code.

All warehouses supported by Monte Carlo work with the monitoring advisor. The skill validates table and column references against your actual warehouse schema via the Monte Carlo API.

## Prerequisites

- Claude Code, Cursor, VS Code or any editor with MCP support
- Monte Carlo account with Editor role or above
- [MC CLI](https://docs.getmontecarlo.com/docs/using-the-cli) installed for monitor deployment (`pip install montecarlodata`)
- **Recommended:** Install alongside the **monitor-creation** skill — the monitoring advisor cross-references its parameter docs

## Setup

### Via the mc-agent-toolkit plugin (recommended)

Install the plugin for your editor — it bundles the skill, hooks, MCP server, and permissions automatically. See the [main README](../../README.md#installing-the-plugin-recommended) for editor-specific instructions.

### Standalone

1. Configure the Monte Carlo MCP server:
```
claude mcp add --transport http monte-carlo-mcp https://integrations.getmontecarlo.com/mcp
```

2. Install the skill:
```bash
npx skills add monte-carlo-data/mc-agent-toolkit --skill monitoring-advisor
```

3. Authenticate: run `/mcp` in your editor, select `monte-carlo-mcp`, and complete the OAuth flow.

4. Verify: ask your editor "Test my Monte Carlo connection" — it should call `testConnection` and confirm.

<details>
<summary>Legacy: header-based auth (for MCP clients without HTTP transport)</summary>

If your MCP client doesn't support HTTP transport, use `.mcp.json.example` with `npx mcp-remote` and header-based authentication. See the [MCP server docs](https://docs.getmontecarlo.com/docs/mcp-server) for details.

</details>

## How to use it

Ask your AI editor about your monitoring coverage — describe what you want to understand or protect. The skill guides the agent through warehouse discovery, use-case analysis, coverage gap identification, and monitor creation. No special commands needed.

### Example prompts

- "What are my coverage gaps?"
- "Show me my use cases and what's monitored"
- "Which tables should I monitor first?"
- "Analyze monitoring coverage for my warehouse"
- "Find unmonitored tables with recent anomalies"
- "Help me set up monitoring for my critical use cases"

### What it does

1. **Discovers** your warehouses and use cases
2. **Analyzes** coverage — which tables are monitored, which aren't, and which have active anomalies
3. **Prioritizes** gaps by criticality, importance score, and anomaly activity
4. **Suggests** monitors tailored to your use cases and data patterns
5. **Generates** monitors-as-code YAML ready for deployment

### Deploying generated monitors

When the advisor generates a monitor, it returns MaC YAML. Deploy with:

```bash
montecarlo monitors apply --dry-run # preview
montecarlo monitors apply --auto-yes # apply
```

Your project needs a `montecarlo.yml` config in the working directory:

```yaml
version: 1
namespace: <your-namespace>
default_resource: <your-warehouse-name>
```
Loading
Loading