feat: harden crawler/parser pipeline and expand plugin coverage

.gitignore

@@ -19,4 +19,5 @@ Requisitos.md
 crystalline-imagining-forest.md
 .ideas/
 .dev/
+.productivity/
 /.venv-wsl

AGENTS.md

@@ -54,7 +54,8 @@ Every agent must follow this **10-step cycle** for every task:
     *   Pick a CLI tool *not yet tested* (check D:\GitHub\cli-plugins\output) and available in WIN or WSL environment.
     *   Run `cli-crawler <tool>` -> `generate-plugin <tool>`.
     *   Verify the output manually in D:\GitHub\cli-plugins\output and D:\GitHub\cli-plugins\plugins to prove the solution works in the wild.
-8.  **MARK TASK**: Check off `[x]` in `specs/001-cli-plugins-base/tasks.md` (or relevant task file).
+    *   Verify the quality of the output and the generated plugin against the main goal of the project. If the quality is not good, go back to step 5 ou defer creating new improvement tasks in `specs/001-cli-plugins-base/tasks.md` and `CLAUDE.local.md`.
+8.  **MARK TASK**: Check off `[x]` in `specs/001-cli-plugins-base/tasks.md` and `CLAUDE.local.md` (or relevant task files).
 9.  **ATOMIC COMMIT**: Commit *only* related changes with a conventional message.
 10. **PUSH + PR**: Push to branch, open PR (if applicable), and **WAIT FOR REVIEW**.
 

README.md

@@ -99,6 +99,11 @@ clis:
       discovery_command: my-tool plugin list
 ```
 
+**Minimal override policy (recommended)**:
+- Keep only operational overrides that differ from defaults: `environment`, `help_pattern`, `max_depth`, `max_concurrent`, `plugins.discovery_command`.
+- Do **not** mirror inventory in `config.yaml` (avoid listing every crawled/generated CLI just to keep a catalog).
+- Use `output/*.json` and `plugins/cli-*` as inventory sources of truth.
+
 ### Step 2: Crawl
 
 ```bash
@@ -229,6 +234,27 @@ uv run python scripts/generate_plugin.py output/docker.json
 uv run python cli_crawler.py --all --config config.yaml
 ```
 
+### Config Audit (Inventory Drift)
+
+Audit drift between `config.yaml`, `output/*.json`, and `plugins/cli-*`:
+
+```bash
+config-audit --config config.yaml --output-dir output --plugins-dir plugins --report output/config-audit.json
+```
+
+Alternative without installing console scripts:
+
+```bash
+uv run python -m config.audit --config config.yaml --output-dir output --plugins-dir plugins --report output/config-audit.json
+```
+
+This report includes:
+- `missing_in_config`
+- `stale_in_config`
+- `missing_output`
+- `missing_plugin`
+- `suggested_minimal_overrides`
+
 ## Design Decisions
 
 - **Python stdlib only** -- No external dependencies for crawler or generator

config.yaml

@@ -4,7 +4,7 @@ defaults:
   max_concurrent: 5
   retry: 1
   environment: wsl
-  raw_threshold: 10240
+  raw_threshold: 10000
 
 clis:
   git:

plugins/cli-awk/.claude-plugin/plugin.json

@@ -0,0 +1,10 @@
+{
+  "name": "cli-awk",
+  "version": "",
+  "description": "Command reference plugin for awk CLI",
+  "keywords": [
+    "awk"
+  ],
+  "repository": "https://github.com/nsalvacao/cli-plugins",
+  "license": "MIT"
+}

plugins/cli-awk/commands/scan-cli.md

@@ -0,0 +1,24 @@
+---
+name: scan-cli
+description: Re-scan the awk CLI and regenerate plugin reference files
+allowed-tools: ["Bash"]
+---
+
+# Re-scan awk CLI
+
+Run the rescan script to crawl the CLI and regenerate this plugin:
+
+```bash
+bash $CLAUDE_PLUGIN_ROOT/scripts/rescan.sh
+```
+
+Add `--dry-run` to preview without writing files:
+
+```bash
+bash $CLAUDE_PLUGIN_ROOT/scripts/rescan.sh --dry-run
+```
+
+## Notes
+
+- Requires `awk` installed and on PATH
+- Idempotent -- re-running overwrites existing files cleanly

plugins/cli-awk/scripts/rescan.sh

@@ -0,0 +1,26 @@
+#!/usr/bin/env bash
+# Re-scan awk CLI and regenerate this plugin.
+# Usage: bash scripts/rescan.sh [--dry-run]
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_ROOT="$(cd "$SCRIPT_DIR/../../.." && pwd)"
+
+CLI_NAME="awk"
+JSON_PATH="$PROJECT_ROOT/output/$CLI_NAME.json"
+PLUGIN_DIR="$PROJECT_ROOT/plugins/cli-awk"
+
+# Check CLI is available
+if ! command -v "$CLI_NAME" &>/dev/null; then
+    echo "ERROR: $CLI_NAME not found on PATH. Install it first." >&2
+    exit 1
+fi
+
+echo "==> Crawling $CLI_NAME..."
+python3 "$PROJECT_ROOT/cli_crawler.py" "$CLI_NAME"
+
+echo "==> Generating plugin..."
+python3 "$PROJECT_ROOT/scripts/generate_plugin.py" "$JSON_PATH" "$@"
+
+echo "==> Done. Plugin at: $PLUGIN_DIR"
+ls -la "$PLUGIN_DIR"

plugins/cli-awk/skills/cli-awk/SKILL.md

@@ -0,0 +1,69 @@
+---
+name: cli-awk
+description: >-
+  This skill should be used when the user needs help with awk CLI commands, flags, and troubleshooting.
+---
+
+# awk CLI Reference
+
+Compact command reference for **awk** v.
+
+- **0** total commands
+- **0** command flags + **28** global flags
+- **0** extracted usage examples
+- Max nesting depth: 0
+
+## When to Use
+
+- Constructing or validating `awk` commands
+- Looking up flags/options fast
+- Troubleshooting failed invocations
+
+## Top-Level Commands
+
+Command format examples: 
+
+### Global Flags
+
+| Flag | Short | Type | Description |
+| --- | --- | --- | --- |
+| `--assign` | `-v` | string | assign=var=val |
+| `--bignum` | `-M` | bool | bignum |
+| `--characters-as-bytes` | `-b` | bool | characters-as-bytes |
+| `--copyright` | `-C` | bool | copyright |
+| `--debug` | `-D` | string | debug[=file] |
+| `--dump-variables` | `-d` | string | dump-variables[=file] |
+| `--exec` | `-E` | string | exec=file |
+| `--field-separator` | `-F` | string | field-separator=fs |
+| `--file` | `-f` | string | file=progfile |
+| `--gen-pot` | `-g` | bool | gen-pot |
+| `--help` | `-h` | bool | help |
+| `--include` | `-i` | string | include=includefile |
+| `--lint` | `-L` | string | lint[=fatal|invalid|no-ext] |
+| `--lint-old` | `-t` | bool | lint-old |
+| `--load` | `-l` | string | load=library |
+| `--no-optimize` | `-s` | bool | no-optimize |
+| `--non-decimal-data` | `-n` | bool | non-decimal-data |
+| `--optimize` | `-O` | bool | optimize |
+| `--posix` | `-P` | bool | posix |
+| `--pretty-print` | `-o` | string | pretty-print[=file] |
+| `--profile` | `-p` | string | profile[=file] |
+| `--re-interval` | `-r` | bool | re-interval |
+| `--sandbox` | `-S` | bool | sandbox |
+| `--source` | `-e` | string | source='program-text' |
+| `--trace` | `-I` | bool | trace |
+| `--traditional` | `-c` | bool | traditional |
+| `--use-lc-numeric` | `-N` | bool | use-lc-numeric |
+| `--version` | `-V` | bool | version |
+
+## Common Usage Patterns (Compact)
+
+_No examples extracted._
+## Detailed References
+
+- Full command tree: `references/commands.md`
+- Full examples catalog: `references/examples.md`
+
+## Re-Scanning
+
+After a CLI update, run `/scan-cli` or execute crawler + generator again.

plugins/cli-awk/skills/cli-awk/references/commands.md

@@ -0,0 +1,35 @@
+# awk -- Complete Command Reference
+
+## Global Flags
+
+| Flag | Short | Type | Description |
+| --- | --- | --- | --- |
+| `--assign` | `-v` | string | assign=var=val |
+| `--bignum` | `-M` | bool | bignum |
+| `--characters-as-bytes` | `-b` | bool | characters-as-bytes |
+| `--copyright` | `-C` | bool | copyright |
+| `--debug` | `-D` | string | debug[=file] |
+| `--dump-variables` | `-d` | string | dump-variables[=file] |
+| `--exec` | `-E` | string | exec=file |
+| `--field-separator` | `-F` | string | field-separator=fs |
+| `--file` | `-f` | string | file=progfile |
+| `--gen-pot` | `-g` | bool | gen-pot |
+| `--help` | `-h` | bool | help |
+| `--include` | `-i` | string | include=includefile |
+| `--lint` | `-L` | string | lint[=fatal|invalid|no-ext] |
+| `--lint-old` | `-t` | bool | lint-old |
+| `--load` | `-l` | string | load=library |
+| `--no-optimize` | `-s` | bool | no-optimize |
+| `--non-decimal-data` | `-n` | bool | non-decimal-data |
+| `--optimize` | `-O` | bool | optimize |
+| `--posix` | `-P` | bool | posix |
+| `--pretty-print` | `-o` | string | pretty-print[=file] |
+| `--profile` | `-p` | string | profile[=file] |
+| `--re-interval` | `-r` | bool | re-interval |
+| `--sandbox` | `-S` | bool | sandbox |
+| `--source` | `-e` | string | source='program-text' |
+| `--trace` | `-I` | bool | trace |
+| `--traditional` | `-c` | bool | traditional |
+| `--use-lc-numeric` | `-N` | bool | use-lc-numeric |
+| `--version` | `-V` | bool | version |
+

plugins/cli-awk/skills/cli-awk/references/examples.md

@@ -0,0 +1,2 @@
+# awk -- Usage Examples
+

plugins/cli-curl/.claude-plugin/plugin.json

@@ -0,0 +1,10 @@
+{
+  "name": "cli-curl",
+  "version": "8.5.0",
+  "description": "Command reference plugin for curl CLI",
+  "keywords": [
+    "curl"
+  ],
+  "repository": "https://github.com/nsalvacao/cli-plugins",
+  "license": "MIT"
+}

plugins/cli-curl/commands/scan-cli.md

@@ -0,0 +1,24 @@
+---
+name: scan-cli
+description: Re-scan the curl CLI and regenerate plugin reference files
+allowed-tools: ["Bash"]
+---
+
+# Re-scan curl CLI
+
+Run the rescan script to crawl the CLI and regenerate this plugin:
+
+```bash
+bash $CLAUDE_PLUGIN_ROOT/scripts/rescan.sh
+```
+
+Add `--dry-run` to preview without writing files:
+
+```bash
+bash $CLAUDE_PLUGIN_ROOT/scripts/rescan.sh --dry-run
+```
+
+## Notes
+
+- Requires `curl` installed and on PATH
+- Idempotent -- re-running overwrites existing files cleanly

plugins/cli-curl/scripts/rescan.sh

@@ -0,0 +1,26 @@
+#!/usr/bin/env bash
+# Re-scan curl CLI and regenerate this plugin.
+# Usage: bash scripts/rescan.sh [--dry-run]
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_ROOT="$(cd "$SCRIPT_DIR/../../.." && pwd)"
+
+CLI_NAME="curl"
+JSON_PATH="$PROJECT_ROOT/output/$CLI_NAME.json"
+PLUGIN_DIR="$PROJECT_ROOT/plugins/cli-$CLI_NAME"
+
+# Check CLI is available
+if ! command -v "$CLI_NAME" &>/dev/null; then
+    echo "ERROR: $CLI_NAME not found on PATH. Install it first." >&2
+    exit 1
+fi
+
+echo "==> Crawling $CLI_NAME..."
+python3 "$PROJECT_ROOT/cli_crawler.py" "$CLI_NAME"
+
+echo "==> Generating plugin..."
+python3 "$PROJECT_ROOT/scripts/generate_plugin.py" "$JSON_PATH" "$@"
+
+echo "==> Done. Plugin at: $PLUGIN_DIR"
+ls -la "$PLUGIN_DIR"

plugins/cli-curl/skills/cli-curl/SKILL.md

@@ -0,0 +1,53 @@
+---
+name: cli-curl
+description: >-
+  This skill should be used when the user needs help with curl CLI commands, flags, and troubleshooting.
+---
+
+# curl CLI Reference
+
+Compact command reference for **curl** v8.5.0.
+
+- **0** total commands
+- **0** command flags + **12** global flags
+- **0** extracted usage examples
+- Max nesting depth: 0
+
+## When to Use
+
+- Constructing or validating `curl` commands
+- Looking up flags/options fast
+- Troubleshooting failed invocations
+
+## Top-Level Commands
+
+Command format examples: 
+
+### Global Flags
+
+| Flag | Short | Type | Description |
+| --- | --- | --- | --- |
+| `--data` | `-d` | string | HTTP POST data |
+| `--fail` | `-f` | bool | Fail fast with no output on HTTP errors |
+| `--help` | `-h` | string | Get help for commands |
+| `--include` | `-i` | bool | Include protocol response headers in the output |
+| `--output` | `-o` | string | Write to file instead of stdout |
+| `--remote-name` | `-O` | bool | Write output to a file named as the remote file |
+| `--silent` | `-s` | bool | Silent mode |
+| `--upload-file` | `-T` | string | Transfer local FILE to destination |
+| `--user` | `-u` | string | Server user and password |
+| `--user-agent` | `-A` | string | Send User-Agent <name> to server |
+| `--verbose` | `-v` | bool | Make the operation more talkative |
+| `--version` | `-V` | bool | Show version number and quit |
+
+## Common Usage Patterns (Compact)
+
+_No examples extracted._
+## Detailed References
+
+- Full command tree: `references/commands.md`
+- Full examples catalog: `references/examples.md`
+
+## Re-Scanning
+
+After a CLI update, run `/scan-cli` or execute crawler + generator again.

plugins/cli-curl/skills/cli-curl/references/commands.md

@@ -0,0 +1,19 @@
+# curl -- Complete Command Reference
+
+## Global Flags
+
+| Flag | Short | Type | Description |
+| --- | --- | --- | --- |
+| `--data` | `-d` | string | HTTP POST data |
+| `--fail` | `-f` | bool | Fail fast with no output on HTTP errors |
+| `--help` | `-h` | string | Get help for commands |
+| `--include` | `-i` | bool | Include protocol response headers in the output |
+| `--output` | `-o` | string | Write to file instead of stdout |
+| `--remote-name` | `-O` | bool | Write output to a file named as the remote file |
+| `--silent` | `-s` | bool | Silent mode |
+| `--upload-file` | `-T` | string | Transfer local FILE to destination |
+| `--user` | `-u` | string | Server user and password |
+| `--user-agent` | `-A` | string | Send User-Agent <name> to server |
+| `--verbose` | `-v` | bool | Make the operation more talkative |
+| `--version` | `-V` | bool | Show version number and quit |
+

plugins/cli-curl/skills/cli-curl/references/examples.md

@@ -0,0 +1,2 @@
+# curl -- Usage Examples
+