From f1ac236e46204140845e5882ee58e907bcfad0f4 Mon Sep 17 00:00:00 2001 From: aesoft <43991222+aeppling@users.noreply.github.com> Date: Mon, 23 Mar 2026 19:00:52 +0100 Subject: [PATCH 1/5] fix: start folder & docs refacto --- docs/TECHNICAL.md | 268 ++++++++++++++++++++++++++++++++++++++ hooks/README.md | 211 ++++++++++++++++++++++++++++++ hooks/claude/README.md | 22 ++++ hooks/cline/README.md | 7 + hooks/codex/README.md | 7 + hooks/copilot/README.md | 14 ++ hooks/cursor/README.md | 7 + hooks/opencode/README.md | 9 ++ hooks/windsurf/README.md | 7 + src/analytics/README.md | 29 +++++ src/cmds/README.md | 165 +++++++++++++++++++++++ src/cmds/cloud/README.md | 9 ++ src/cmds/dotnet/README.md | 7 + src/cmds/git/README.md | 13 ++ src/cmds/go/README.md | 7 + src/cmds/js/README.md | 13 ++ src/cmds/python/README.md | 13 ++ src/cmds/ruby/README.md | 9 ++ src/cmds/rust/README.md | 7 + src/cmds/system/README.md | 12 ++ src/core/README.md | 124 ++++++++++++++++++ src/hooks/README.md | 83 ++++++++++++ 22 files changed, 1043 insertions(+) create mode 100644 docs/TECHNICAL.md create mode 100644 hooks/README.md create mode 100644 hooks/claude/README.md create mode 100644 hooks/cline/README.md create mode 100644 hooks/codex/README.md create mode 100644 hooks/copilot/README.md create mode 100644 hooks/cursor/README.md create mode 100644 hooks/opencode/README.md create mode 100644 hooks/windsurf/README.md create mode 100644 src/analytics/README.md create mode 100644 src/cmds/README.md create mode 100644 src/cmds/cloud/README.md create mode 100644 src/cmds/dotnet/README.md create mode 100644 src/cmds/git/README.md create mode 100644 src/cmds/go/README.md create mode 100644 src/cmds/js/README.md create mode 100644 src/cmds/python/README.md create mode 100644 src/cmds/ruby/README.md create mode 100644 src/cmds/rust/README.md create mode 100644 src/cmds/system/README.md create mode 100644 src/core/README.md create mode 100644 src/hooks/README.md diff --git a/docs/TECHNICAL.md b/docs/TECHNICAL.md new file mode 100644 index 00000000..9588f018 --- /dev/null +++ b/docs/TECHNICAL.md @@ -0,0 +1,268 @@ +# RTK Technical Documentation + +> For contributors and maintainers. See `CLAUDE.md` for development commands and coding guidelines. +> Each folder has its own README.md with implementation details, file descriptions, and contribution guidelines. + +--- + +## 1. Project Vision + +LLM-powered coding agents (Claude Code, Copilot, Cursor, etc.) consume tokens for every CLI command output they process. Most command outputs contain boilerplate, progress bars, ANSI escape codes, and verbose formatting that wastes tokens without providing actionable information. + +RTK sits between the agent and the CLI, filtering outputs to keep only what matters. This achieves 60-90% token savings per command, reducing costs and increasing effective context window utilization. RTK is a single Rust binary with zero external dependencies, adding less than 10ms overhead per command. + +--- + +## 2. Architecture Overview + +``` +User / LLM Agent + | + v ++--------------------------------------------------+ +| LLM Agent Hook | +| hooks/{claude,copilot,cursor,...}/ | +| Intercepts: "git status" -> "rtk git status" | ++-------------------------+------------------------+ + | + v ++--------------------------------------------------+ +| RTK CLI (main.rs) | +| | +| +-------------+ +-----------------+ | +| | Clap Parser | -> | Command Routing | | +| | (Commands | | (match on enum) | | +| | enum) | +--------+--------+ | +| +-------------+ | | +| +---------+---------+ | +| v v v | +| +----------+ +--------+ +----------+| +| |Rust Filter| |TOML DSL| |Passthru || +| |(cmds/**) | |Filter | |(fallback)|| +| +-----+----+ +----+---+ +----+-----+| +| | | | | +| +-----+-----+-----------+ | +| v | +| +---------------------+ | +| | Token Tracking | | +| | (core/tracking) | | +| | SQLite DB | | +| +---------------------+ | ++--------------------------------------------------+ +``` + +**Design principles:** +- Single-threaded, no async (startup < 10ms) +- Graceful degradation: filter failure falls back to raw output +- Exit code propagation: RTK never swallows non-zero exits +- Transparent proxy: unknown commands pass through unchanged + +--- + +## 3. End-to-End Flow + +This is the full lifecycle of a command through RTK, from LLM agent to filtered output. + +### 3.1 Hook Installation (`rtk init`) + +The user runs `rtk init` to set up hooks for their LLM agent. This: + +1. Writes a thin shell hook script (e.g., `~/.claude/hooks/rtk-rewrite.sh`) +2. Stores its SHA-256 hash for integrity verification +3. Patches the agent's settings file (e.g., `settings.json`) to register the hook +4. Writes RTK awareness instructions (e.g., `RTK.md`) for prompt-level guidance + +RTK supports 7 agents, each with its own installation mode. The hook scripts are embedded in the binary and written at install time. + +> **Details**: [`src/hooks/README.md`](../src/hooks/README.md) covers all installation modes, configuration files, and the uninstall flow. + +### 3.2 Hook Interception (Command Rewriting) + +When an LLM agent runs a command (e.g., `git status`): + +1. The agent fires a `PreToolUse` event (or equivalent) containing the command as JSON +2. The hook script reads the JSON, extracts the command string +3. The hook calls `rtk rewrite "git status"` as a subprocess +4. `rtk rewrite` consults the command registry (70+ patterns) and returns `rtk git status` +5. The hook sends a response telling the agent to use the rewritten command +6. If anything fails (jq missing, rtk not found, no match), the hook exits silently -- the raw command runs unchanged + +All rewrite logic lives in Rust (`src/discover/registry.rs`). Hooks are thin delegates that handle agent-specific JSON formats. + +> **Details**: [`hooks/README.md`](../hooks/README.md) covers each agent's JSON format, the rewrite registry, compound command handling, and the `RTK_DISABLED` override. + +### 3.3 CLI Parsing and Routing + +Once the rewritten command reaches RTK: + +1. **Telemetry**: `telemetry::maybe_ping()` fires a non-blocking daily usage ping +2. **Clap parsing**: `Cli::try_parse()` matches against the `Commands` enum (~50 subcommands) +3. **Hook check**: `hook_check::maybe_warn()` warns if the installed hook is outdated (rate-limited to 1/day) +4. **Integrity check**: `integrity::runtime_check()` verifies the hook's SHA-256 hash for operational commands +5. **Routing**: A `match cli.command` dispatches to the specialized filter module + +If Clap parsing fails (command not in the enum), the fallback path runs instead. + +### 3.4 Filter Execution + +RTK has two filter systems: + +**Rust Filters** (~45 commands): Compiled modules in `src/cmds/` that execute the command, parse its output, and apply specialized transformations (regex, JSON, state machines). + +**TOML DSL Filters** (~60 built-in): Declarative filters in `src/filters/*.toml` that apply regex-based line filtering, truncation, and section extraction. Applied in `run_fallback()` when no Rust filter matches. + +Each filter module follows the same pattern: +1. Start a timer (`TimedExecution::start()`) +2. Execute the underlying command (`std::process::Command`) +3. Apply filtering (strip boilerplate, group errors, truncate) +4. On filter error, fall back to raw output +5. Track token savings to SQLite +6. Propagate exit code + +> **Details**: [`src/cmds/README.md`](../src/cmds/README.md) covers the common pattern, ecosystem organization, cross-command dependencies, and how to add new filters. + +### 3.5 Fallback Path + +When Clap parsing fails (unknown command): + +1. Guard: check if the command is an RTK meta-command (`gain`, `init`, etc.) -- if so, show Clap error +2. Look up TOML DSL filters via `toml_filter::find_matching_filter()` +3. If TOML match: capture stdout, apply filter pipeline, track savings +4. If no match: pure passthrough with `Stdio::inherit`, track as 0% savings + +``` +Command received + -> Clap parse succeeds? + -> Yes: Route to Rust filter module + -> No: run_fallback() + -> TOML filter match? + -> Yes: Capture stdout, apply filter, track savings + -> No: Passthrough (inherit stdio, track 0% savings) +``` + +> **Details**: [`src/core/README.md`](../src/core/README.md) covers the TOML filter engine, filter pipeline stages, and trust-gated project filters. + +### 3.6 Token Tracking + +Every command execution records metrics to SQLite (`~/.local/share/rtk/tracking.db`): + +- Input tokens (raw output size) and output tokens (filtered size) +- Savings percentage, execution time, project path +- 90-day automatic retention cleanup +- Token estimation: `ceil(chars / 4.0)` approximation + +Analytics commands (`rtk gain`, `rtk cc-economics`, `rtk session`) query this database to produce dashboards and ROI reports. + +> **Details**: [`src/analytics/README.md`](../src/analytics/README.md) covers the analytics modules, and [`src/core/README.md`](../src/core/README.md) covers the tracking database schema. + +### 3.7 Tee Recovery + +On command failure (non-zero exit code): + +1. Raw unfiltered output is saved to `~/.local/share/rtk/tee/{epoch}_{slug}.log` +2. A hint line is printed: `[full output: ~/.../tee/1234_cargo_test.log]` +3. LLM agents can re-read the file instead of re-running the failed command + +Tee is configurable (enabled/disabled, min size, max files, max file size) and never affects command output or exit code on failure. + +> **Details**: [`src/core/README.md`](../src/core/README.md) covers tee configuration and the rotation strategy. + +--- + +## 4. Folder Map + +``` +src/ ++-- main.rs # CLI entry point, Commands enum, command routing ++-- core/ # Shared infrastructure (8 files) +| +-- README.md # -> Details: tracking, config, tee, TOML filters ++-- hooks/ # Hook system (8 files) +| +-- README.md # -> Details: init, integrity, rewrite, verify ++-- analytics/ # Token savings analytics (4 files) +| +-- README.md # -> Details: gain, economics, session ++-- cmds/ # Command filter modules (45 files) +| +-- README.md # -> Details: common pattern, ecosystem organization +| +-- git/ # Git + GitHub CLI + Graphite (4 files) +| +-- rust/ # Cargo + runner (2 files) +| +-- js/ # JS/TS/Node ecosystem (9 files) +| +-- python/ # Python ecosystem (4 files) +| +-- go/ # Go ecosystem (2 files) +| +-- dotnet/ # .NET ecosystem (4 files) +| +-- cloud/ # Cloud and infra (5 files) +| +-- system/ # System utilities (13 files) ++-- discover/ # Claude Code history analysis ++-- learn/ # CLI correction detection ++-- parser/ # Parser infrastructure ++-- filters/ # 60+ TOML filter configs + +hooks/ # LLM agent hook scripts (root directory) ++-- README.md # -> Details: agent JSON formats, rewrite flow ++-- claude/ # Claude Code (shell hook) ++-- copilot/ # GitHub Copilot (Rust binary hook) ++-- cursor/ # Cursor IDE (shell hook) ++-- cline/ # Cline / Roo Code (rules file) ++-- windsurf/ # Windsurf / Cascade (rules file) ++-- codex/ # OpenAI Codex CLI (awareness doc) ++-- opencode/ # OpenCode (TypeScript plugin) +``` + +--- + +## 5. Hook System Summary + +RTK supports 7 LLM agents through hook integrations: + +| Agent | Hook Type | Mechanism | Can Modify Command? | +|-------|-----------|-----------|---------------------| +| Claude Code | Shell hook | `PreToolUse` in `settings.json` | Yes (`updatedInput`) | +| GitHub Copilot (VS Code) | Rust binary | `rtk hook copilot` reads JSON | Yes (`updatedInput`) | +| GitHub Copilot CLI | Rust binary | `rtk hook copilot` reads JSON | No (deny + suggestion) | +| Cursor | Shell hook | `preToolUse` hook | Yes (`updated_input`) | +| Gemini CLI | Rust binary | `rtk hook gemini` reads JSON | Yes (`hookSpecificOutput`) | +| Cline/Roo Code | Rules file | Prompt-level guidance | N/A (prompt) | +| Windsurf | Rules file | Prompt-level guidance | N/A (prompt) | +| Codex CLI | Awareness doc | AGENTS.md integration | N/A (prompt) | +| OpenCode | TS plugin | `tool.execute.before` event | Yes (in-place mutation) | + +> **Details**: [`hooks/README.md`](../hooks/README.md) has the full JSON schemas for each agent. [`src/hooks/README.md`](../src/hooks/README.md) covers installation, integrity verification, and the rewrite command. + +--- + +## 6. Filter Pipeline Summary + +### Rust Filters (cmds/**) + +Compiled filter modules for complex transformations. ~45 commands with 60-95% token savings. + +> **Details**: [`src/cmds/README.md`](../src/cmds/README.md) and each ecosystem subdirectory README. + +### TOML DSL Filters (src/filters/*.toml) + +Declarative filters with an 8-stage pipeline: strip ANSI, regex replace, match output, strip/keep lines, truncate lines, head/tail, max lines, on-empty message. Loaded from three tiers: built-in (compiled), global (`~/.config/rtk/filters/`), project-local (`.rtk/filters/`, trust-gated). + +> **Details**: [`src/core/README.md`](../src/core/README.md) covers the TOML filter engine. + +--- + +## 7. Performance Constraints + +| Metric | Target | Verification | +|--------|--------|--------------| +| Startup time | < 10ms | `hyperfine 'rtk git status' 'git status'` | +| Memory usage | < 5MB resident | `/usr/bin/time -v rtk git status` | +| Binary size | < 5MB stripped | `ls -lh target/release/rtk` | +| Token savings | 60-90% per filter | Snapshot + token count tests | + +Achieved through: +- Zero async overhead (single-threaded, no tokio) +- Lazy regex compilation (`lazy_static!`) +- Minimal allocations (borrow over clone) +- No config file I/O on startup (loaded on-demand) + +--- + +## 8. Future Improvements + +- **Extract cli.rs**: Move `Commands` enum, 13 sub-enums (`GitCommands`, `CargoCommands`, etc.), and `AgentTarget` from main.rs to a dedicated cli.rs module. This would reduce main.rs from ~2600 to ~1500 lines. +- **Split routing**: Extract the `match cli.command { ... }` block into a separate routing module. +- **Streaming filters**: For long-running commands, filter output line-by-line as it arrives instead of buffering. diff --git a/hooks/README.md b/hooks/README.md new file mode 100644 index 00000000..a0b0265c --- /dev/null +++ b/hooks/README.md @@ -0,0 +1,211 @@ +# LLM Agent Hooks + +## Scope + +**Deployed hook artifacts** — the actual files installed on user machines by `rtk init`. These are shell scripts, TypeScript plugins, and rules files that run outside the Rust binary. They are **thin delegates**: parse agent-specific JSON, call `rtk rewrite` as a subprocess, format agent-specific response. Zero filtering logic lives here. + +Owns: per-agent hook scripts and configuration files for 7 supported agents (Claude Code, Copilot, Cursor, Cline, Windsurf, Codex, OpenCode). + +Does **not** own: hook installation/uninstallation (that's `src/hooks/init.rs`), the rewrite pattern registry (that's `discover/registry`), or integrity verification (that's `src/hooks/integrity.rs`). + +Relationship to `src/hooks/`: that component **creates** these files; this directory **contains** them. + +## Purpose + +LLM agent integrations that intercept CLI commands and route them through RTK for token optimization. Each hook transparently rewrites raw commands (e.g., `git status`) to their RTK equivalents (e.g., `rtk git status`), delivering 60-90% token savings without requiring the agent or user to change their workflow. + +## How It Works + +``` +Agent runs command (e.g., "cargo test --nocapture") + -> Hook intercepts (PreToolUse / plugin event) + -> Reads JSON input, extracts command string + -> Calls `rtk rewrite "cargo test --nocapture"` + -> Registry matches pattern, returns "rtk cargo test --nocapture" + -> Hook sends response in agent-specific JSON format + -> Agent executes "rtk cargo test --nocapture" instead + -> Filtered output reaches LLM (~90% fewer tokens) +``` + +All rewrite logic lives in the Rust binary (`src/discover/registry.rs`). Hook scripts are **thin delegates** that handle agent-specific JSON formats and call `rtk rewrite` for the actual decision. This ensures a single source of truth for all 70+ rewrite patterns. + +## Directory Structure + +``` +hooks/ + claude/ Claude Code (shell hook + settings.json) + copilot/ GitHub Copilot (VS Code Chat + Copilot CLI) + cursor/ Cursor IDE (shell hook) + cline/ Cline / Roo Code VS Code extension (rules file) + windsurf/ Windsurf / Cascade IDE (rules file) + codex/ OpenAI Codex CLI (awareness file) + opencode/ OpenCode (TypeScript plugin) +``` + +## Supported Agents + +| Agent | Mechanism | Hook Type | Can Modify Command? | +|-------|-----------|-----------|---------------------| +| Claude Code | Shell hook (`PreToolUse`) | Transparent rewrite | Yes (`updatedInput`) | +| VS Code Copilot Chat | Rust binary (`rtk hook copilot`) | Transparent rewrite | Yes (`updatedInput`) | +| GitHub Copilot CLI | Rust binary (`rtk hook copilot`) | Deny-with-suggestion | No (agent retries) | +| Cursor | Shell hook (`preToolUse`) | Transparent rewrite | Yes (`updated_input`) | +| Gemini CLI | Rust binary (`rtk hook gemini`) | Transparent rewrite | Yes (`hookSpecificOutput`) | +| Cline / Roo Code | Custom instructions (rules file) | Prompt-level guidance | N/A | +| Windsurf | Custom instructions (rules file) | Prompt-level guidance | N/A | +| Codex CLI | AGENTS.md / instructions | Prompt-level guidance | N/A | +| OpenCode | TypeScript plugin (`tool.execute.before`) | In-place mutation | Yes | + +## JSON Formats by Agent + +### Claude Code (Shell Hook) + +**Input** (stdin): +```json +{ + "tool_name": "Bash", + "tool_input": { "command": "git status" } +} +``` + +**Output** (stdout, when rewritten): +```json +{ + "hookSpecificOutput": { + "hookEventName": "PreToolUse", + "permissionDecision": "allow", + "permissionDecisionReason": "RTK auto-rewrite", + "updatedInput": { "command": "rtk git status" } + } +} +``` + +### Cursor (Shell Hook) + +**Input**: Same as Claude Code. + +**Output** (stdout, when rewritten): +```json +{ + "permission": "allow", + "updated_input": { "command": "rtk git status" } +} +``` + +Returns `{}` when no rewrite (Cursor requires JSON for all paths). + +### Copilot CLI (Rust Binary) + +**Input** (stdin, camelCase, `toolArgs` is JSON-stringified): +```json +{ + "toolName": "bash", + "toolArgs": "{\"command\": \"git status\"}" +} +``` + +**Output** (no `updatedInput` support -- uses deny-with-suggestion): +```json +{ + "permissionDecision": "deny", + "permissionDecisionReason": "Token savings: use `rtk git status` instead" +} +``` + +### VS Code Copilot Chat (Rust Binary) + +**Input** (stdin, snake_case): +```json +{ + "tool_name": "Bash", + "tool_input": { "command": "git status" } +} +``` + +**Output**: Same as Claude Code format (with `updatedInput`). + +### Gemini CLI (Rust Binary) + +**Input** (stdin): +```json +{ + "tool_name": "run_shell_command", + "tool_input": { "command": "git status" } +} +``` + +**Output** (when rewritten): +```json +{ + "decision": "allow", + "hookSpecificOutput": { + "tool_input": { "command": "rtk git status" } + } +} +``` + +**No rewrite**: `{"decision": "allow"}` + +### OpenCode (TypeScript Plugin) + +Mutates `args.command` in-place via the zx library: +```typescript +const result = await $`rtk rewrite ${command}`.quiet().nothrow() +const rewritten = String(result.stdout).trim() +if (rewritten && rewritten !== command) { + (args as Record).command = rewritten +} +``` + +## Command Rewrite Registry + +The registry (`src/discover/registry.rs`) handles 70+ command patterns across these categories: + +| Category | Examples | Savings | +|----------|----------|---------| +| Test Runners | vitest, pytest, cargo test, go test, playwright | 90-99% | +| Build Tools | cargo build, npm, pnpm, dotnet, make | 70-90% | +| VCS | git status/log/diff/show | 70-80% | +| Language Servers | tsc, mypy | 80-83% | +| Linters | eslint, ruff, golangci-lint, biome | 80-85% | +| Package Managers | pip, cargo install, pnpm list | 75-80% | +| File Operations | ls, find, grep, cat, head, tail | 60-75% | +| Infrastructure | docker, kubectl, aws, terraform | 75-85% | + +### Compound Command Handling + +The registry handles `&&`, `||`, `;`, `|`, and `&` operators: + +- **Pipe** (`|`): Only the left side is rewritten (right side consumes output format) +- **And/Or/Semicolon** (`&&`, `||`, `;`): Both sides rewritten independently +- **find/fd in pipes**: Never rewritten (output format incompatible with xargs/wc/grep) + +Example: `cargo fmt --all && cargo test` becomes `rtk cargo fmt --all && rtk cargo test` + +### Override Controls + +- **`RTK_DISABLED=1`**: Per-command override (`RTK_DISABLED=1 git status` runs raw) +- **`exclude_commands`**: In `~/.config/rtk/config.toml`, list commands to never rewrite +- **Already-RTK**: `rtk git status` passes through unchanged (no `rtk rtk git`) + +## Exit Code Contract + +Hooks must **never block command execution**. All error paths (missing binary, bad JSON, rewrite failure) must exit 0 so the agent's command runs unmodified. A hook that exits non-zero prevents the user's command from executing. + +When there is no rewrite to apply, the hook must produce no output (or `{}` for Cursor, which requires JSON on all paths). + +### Gaps (to be fixed) + +- `hook_cmd.rs::run_gemini()` — exits 1 on invalid JSON input instead of exit 0 + +## Graceful Degradation + +Hooks are **non-blocking** -- they never prevent a command from executing: + +- jq not installed: warning to stderr, exit 0 (command runs raw) +- rtk binary not found: warning to stderr, exit 0 +- rtk version too old (< 0.23.0): warning to stderr, exit 0 +- Invalid JSON input: pass through unchanged +- `rtk rewrite` crashes: hook exits 0 (subprocess error ignored) +- Filter logic error: fallback to raw command output + diff --git a/hooks/claude/README.md b/hooks/claude/README.md new file mode 100644 index 00000000..b0cd8f1e --- /dev/null +++ b/hooks/claude/README.md @@ -0,0 +1,22 @@ +# Claude Code Hooks + +## Specifics + +- Shell-based `PreToolUse` hook -- requires `jq` for JSON parsing +- Returns `updatedInput` JSON for transparent command rewrite (agent doesn't know RTK is involved) +- Exits silently (exit 0) on any failure: jq missing, rtk missing, rtk too old (< 0.23.0), no match +- Version guard checks `rtk --version` against minimum 0.23.0 +- `rtk-awareness.md` is a slim 10-line instructions file embedded into CLAUDE.md by `rtk init` + +## Testing + +```bash +# Run the full test suite (60+ assertions) +bash hooks/test-rtk-rewrite.sh + +# Test against a specific hook path +HOOK=/path/to/rtk-rewrite.sh bash hooks/test-rtk-rewrite.sh + +# Enable audit logging during testing +RTK_HOOK_AUDIT=1 RTK_AUDIT_DIR=/tmp bash hooks/test-rtk-rewrite.sh +``` diff --git a/hooks/cline/README.md b/hooks/cline/README.md new file mode 100644 index 00000000..8f11bc3f --- /dev/null +++ b/hooks/cline/README.md @@ -0,0 +1,7 @@ +# Cline / Roo Code Hooks + +## Specifics + +- Prompt-level guidance only (no programmatic hook) -- relies on Cline reading custom instructions +- `rules.md` contains the instruction to prefix all shell commands with `rtk`, usage examples, and meta commands +- Installed to `.clinerules` (project-local) by `rtk init` diff --git a/hooks/codex/README.md b/hooks/codex/README.md new file mode 100644 index 00000000..9c8362fb --- /dev/null +++ b/hooks/codex/README.md @@ -0,0 +1,7 @@ +# Codex CLI Hooks + +## Specifics + +- Prompt-level guidance via awareness document -- no programmatic hook +- `rtk-awareness.md` is injected into `AGENTS.md` with an `@RTK.md` reference +- Installed to `~/.codex/` by `rtk init --codex` diff --git a/hooks/copilot/README.md b/hooks/copilot/README.md new file mode 100644 index 00000000..16500363 --- /dev/null +++ b/hooks/copilot/README.md @@ -0,0 +1,14 @@ +# GitHub Copilot Hooks + +## Specifics + +- Uses the `rtk hook copilot` Rust binary (not a shell script) -- no `jq` dependency +- Auto-detects two input formats: VS Code Copilot Chat (snake_case `tool_name`/`tool_input`) and Copilot CLI (camelCase `toolName`/`toolArgs` with JSON-stringified args) +- VS Code format: returns `updatedInput` for transparent rewrite +- Copilot CLI format: returns `permissionDecision: "deny"` with suggestion (Copilot CLI API doesn't support `updatedInput`) + +## Testing + +```bash +bash hooks/test-copilot-rtk-rewrite.sh +``` diff --git a/hooks/cursor/README.md b/hooks/cursor/README.md new file mode 100644 index 00000000..467c3b7b --- /dev/null +++ b/hooks/cursor/README.md @@ -0,0 +1,7 @@ +# Cursor IDE Hooks + +## Specifics + +- Same delegating pattern as Claude Code hook but outputs Cursor's JSON format (`permission`/`updated_input` instead of `hookSpecificOutput`/`updatedInput`) +- Returns `{}` (empty JSON) when no rewrite applies -- Cursor requires JSON output for all code paths +- Requires `jq` and `rtk >= 0.23.0` diff --git a/hooks/opencode/README.md b/hooks/opencode/README.md new file mode 100644 index 00000000..94bca2af --- /dev/null +++ b/hooks/opencode/README.md @@ -0,0 +1,9 @@ +# OpenCode Hooks + +## Specifics + +- TypeScript plugin using the zx library (not a shell hook) +- Intercepts `tool.execute.before` events, calls `rtk rewrite` as a subprocess +- Uses `.quiet().nothrow()` to silently ignore failures +- Mutates `args.command` in-place if rewrite differs from original +- Installed to `~/.config/opencode/plugins/rtk.ts` by `rtk init -g --opencode` diff --git a/hooks/windsurf/README.md b/hooks/windsurf/README.md new file mode 100644 index 00000000..c4a3653b --- /dev/null +++ b/hooks/windsurf/README.md @@ -0,0 +1,7 @@ +# Windsurf (Cascade) Hooks + +## Specifics + +- Prompt-level guidance only (no programmatic hook) -- relies on Windsurf Cascade reading rules files +- `rules.md` contains the instruction to prefix commands with `rtk` +- Installed to `.windsurfrules` (project-local, workspace-scoped) by `rtk init` diff --git a/src/analytics/README.md b/src/analytics/README.md new file mode 100644 index 00000000..b75c939a --- /dev/null +++ b/src/analytics/README.md @@ -0,0 +1,29 @@ +# Analytics + +## Scope + +**Read-only dashboards** over the tracking database. Analytics presents the value that `cmds/` creates — it queries token savings, correlates with external spending data, and surfaces adoption opportunities. It never modifies the tracking DB. + +Owns: `rtk gain` (savings dashboard), `rtk cc-economics` (cost reduction), `rtk session` (adoption analysis), and Claude Code usage data parsing. + +Does **not** own: recording token savings (that's `core/tracking` called by `cmds/`), or command filtering itself (that's `cmds/`). + +Boundary rule: if a new module writes to the DB, it belongs in `core/` or `cmds/`, not here. Tool-specific analytics (like `cc_economics` reading Claude Code data) are fine — the boundary is "read-only presentation", not "tool-agnostic". + +## Purpose +Token savings analytics, economic modeling, and adoption metrics. + +These modules read from the SQLite tracking database to produce dashboards, spending estimates, and session-level adoption reports that help users understand the value RTK provides. + + + +## Files +| File | Responsibility | +|------|---------------| +| gain.rs | `rtk gain` command -- token savings dashboard with ASCII graphs, per-command history, quota consumption estimates, and cumulative savings over time; the primary user-facing analytics view | +| cc_economics.rs | `rtk cc-economics` command -- combines Claude Code API spending data with RTK savings to show net cost reduction; correlates token savings with dollar amounts | +| ccusage.rs | Claude Code usage data parser; defines `CcusagePeriod` and `Granularity` types; reads Claude Code session spending records to feed into `cc_economics.rs` | +| session_cmd.rs | `rtk session` command -- per-session RTK adoption analysis; shows which commands in a coding session were routed through RTK vs executed raw, highlighting missed savings opportunities | + +## Adding New Functionality +To add a new analytics view: (1) create a new `*_cmd.rs` file in this directory, (2) query `core/tracking` for the metrics you need using the existing `TrackingDb` API, (3) register the command in `main.rs` under the `Commands` enum, and (4) add `#[cfg(test)]` unit tests with sample tracking data. Analytics modules should be read-only against the tracking database and never modify it. diff --git a/src/cmds/README.md b/src/cmds/README.md new file mode 100644 index 00000000..49f9d5a7 --- /dev/null +++ b/src/cmds/README.md @@ -0,0 +1,165 @@ +# Command Filter Modules + +## Scope + +**Command execution and output filtering** — this is the core value RTK delivers. Every module here calls an external CLI tool (`Command::new("some_tool")`), transforms its stdout/stderr to reduce token consumption, and records savings via `core/tracking`. + +Owns: all command-specific filter logic, organized by ecosystem (git, rust, js, python, go, dotnet, cloud, system). Cross-ecosystem routing (e.g., `lint_cmd` detecting Python and delegating to `ruff_cmd`) is an intra-component concern. + +Does **not** own: the TOML DSL filter engine (that's `core/toml_filter`), hook interception (that's `hooks/`), or analytics dashboards (that's `analytics/`). This component **writes** to the tracking DB; analytics **reads** from it. + +Boundary rule: a module belongs here if and only if it executes an external command and filters its output. Infrastructure that serves multiple modules without calling external commands belongs in `core/`. + +## Purpose +All command-specific filter modules that execute CLI commands and transform their output to minimize LLM token consumption. Each module follows a consistent pattern: execute the underlying command, filter its output through specialized parsers, track token savings, and propagate exit codes. + +## Organization +Commands are organized by ecosystem: + +| Directory | Ecosystem | Commands | +|-----------|-----------|----------| +| `git/` | Git and VCS | git, gh (GitHub CLI), gt (Graphite), diff | +| `rust/` | Rust | cargo (build, test, clippy, check), generic runner | +| `js/` | JavaScript/TypeScript/Node | npm, pnpm, vitest, lint, tsc, next, prettier, playwright, prisma | +| `python/` | Python | ruff, pytest, mypy, pip | +| `go/` | Go | go (test, build, vet), golangci-lint | +| `dotnet/` | .NET | dotnet (build, test, format), TRX/binlog parsers | +| `cloud/` | Cloud and Infrastructure | aws, docker/kubectl, curl, wget, psql | +| `system/` | System and Generic Utilities | ls, tree, read, grep, find, wc, env, json, log, deps, summary, format, smart | + +## Common Pattern + +Every command module follows this structure: + +```rust +pub fn run(args: MyArgs, verbose: u8) -> Result<()> { + // 1. Start timer for tracking + let timer = tracking::TimedExecution::start(); + + // 2. Execute the underlying command + let output = resolved_command("mycmd") + .args(&args.to_cmd_args()) + .output() + .context("Failed to execute mycmd")?; + + let stdout = String::from_utf8_lossy(&output.stdout); + let stderr = String::from_utf8_lossy(&output.stderr); + let raw = format!("{}\n{}", stdout, stderr); + + // 3. Filter the output (with fallback to raw on error) + let filtered = filter_output(&stdout) + .unwrap_or_else(|e| { + eprintln!("rtk: filter warning: {}", e); + raw.clone() // Passthrough on failure + }); + + // 4. Tee raw output on failure (for LLM re-read) + let exit_code = output.status.code().unwrap_or(1); + if let Some(hint) = tee::tee_and_hint(&raw, "mycmd", exit_code) { + println!("{}\n{}", filtered, hint); + } else { + println!("{}", filtered); + } + + // 5. Track token savings to SQLite + timer.track("mycmd args", "rtk mycmd args", &raw, &filtered); + + // 6. Propagate exit code + if !output.status.success() { + std::process::exit(exit_code); + } + Ok(()) +} +``` + +Key aspects of this pattern: +- **`TimedExecution`**: Records elapsed time, estimates tokens (`ceil(chars / 4.0)`), writes to SQLite +- **`resolved_command()`**: Finds the command in PATH, handles aliases +- **Fallback**: Filter errors fall back to raw output (never block the user) +- **Tee recovery**: On failure, saves raw output to disk with a hint line for LLM re-read +- **Exit code**: Always propagated via `std::process::exit(code)` for CI/CD reliability + +## Token Savings by Category + +| Category | Commands | Typical Savings | Strategy | +|----------|----------|----------------|----------| +| Test Runners | vitest, pytest, cargo test, go test, playwright | 90-99% | Show failures only, aggregate passes | +| Build Tools | cargo build, npm, pnpm, dotnet | 70-90% | Strip progress bars, summarize errors | +| VCS | git status/log/diff/show | 70-80% | Compact commit hashes, stat summaries | +| Linters | eslint/biome, ruff, tsc, mypy, golangci-lint | 80-85% | Group by file/rule, strip context | +| Package Managers | pip, cargo install, pnpm list | 75-80% | Remove decorative output, compact trees | +| File Operations | ls, find, grep, cat/head/tail | 60-75% | Tree format, grouped results, truncation | +| Infrastructure | docker, kubectl, aws, terraform | 75-85% | Essential info only | + +## Cross-Command Dependencies + +- `lint_cmd` routes to `mypy_cmd` or `ruff_cmd` when detecting Python projects +- `format_cmd` routes to `prettier_cmd` or `ruff_cmd` depending on the formatter detected +- `gh_cmd` imports markdown filtering helpers from `git` + +## Cross-Cutting Behavior Contracts + +These behaviors must be uniform across all command modules. Full audit details in `docs/ISO_ANALYZE.md`. + +### Exit Code Propagation + +Modules must capture the underlying command's exit code, propagate it via `std::process::exit()` only on failure, and return `Ok(())` on success. When the process is killed by signal (`.code()` returns `None`), default to exit code 1. + +### Filter Failure Passthrough + +When filtering fails, fall back to raw output and warn on stderr. Never block the user. + +### Tee Recovery + +Modules that parse structured output (JSON, NDJSON, state machines) must call `tee::tee_and_hint()` so users can recover full output on failure. + +### Stderr Handling + +Modules must capture stderr and include it in the raw string passed to `timer.track()`, so token savings reflect total output. + +### Tracking Completeness + +All modules must call `timer.track()` on every path — success, failure, and fallback. Never exit before tracking. + +### Verbose Flag + +All modules accept `verbose: u8`. Use it to print debug info (command being run, savings %, filter tier). Do not accept and ignore it. + +### Gaps (to be fixed) + +**Exit code** — 5 different patterns coexist, should be reviewed for uniform behavior: +- `vitest_cmd.rs`, `tsc_cmd.rs`, `psql_cmd.rs` — exit unconditionally, even on success +- `lint_cmd.rs` — swallows signal kills silently +- `golangci_cmd.rs` — maps signal kill to exit 130 (correct but unique) + +**Filter passthrough** — silent passthrough, no warning: +- `gh_cmd.rs`, `pip_cmd.rs`, `container.rs`, `dotnet_cmd.rs` — `run_passthrough()` skips filtering without warning +- `pnpm_cmd.rs`, `playwright_cmd.rs` — 3-tier degradation but no tee recovery on final tier + +**Tee recovery** — currently 12/38 modules implement tee. Missing from high-risk modules: +- `pnpm_cmd.rs`, `playwright_cmd.rs` — 3-tier parsers, no tee +- `gh_cmd.rs` — aggressive markdown filtering, no tee +- `ruff_cmd.rs`, `golangci_cmd.rs` — JSON parsers, no tee +- `psql_cmd.rs` — has tee but exits before calling it on error path + +**Stderr handling** — 3 patterns coexist. Some modules combine stderr into raw (correct), others print via `eprintln!()` and exclude from tracking (inflates savings %). See `docs/ISO_ANALYZE.md` section 4. + +**Tracking** — exit before track on error path: +- `ls.rs`, `tree.rs` — lost metrics on failure +- `container.rs` — inconsistent across subcommands + +**Verbose** — accept parameter but ignore it: +- `container.rs` — all internal functions prefix `_verbose` +- `diff_cmd.rs` — `_verbose` unused + +## Adding a New Command Filter + +1. Create `_cmd.rs` in the appropriate ecosystem subdirectory +2. Follow the common pattern above (timer, execute, filter with fallback, tee, track, exit code) +3. Use `lazy_static!` for all regex patterns +4. Add the command to the `Commands` enum in `main.rs` +5. Create a test fixture in `tests/fixtures/` from real command output +6. Write snapshot test (`assert_snapshot!`) and token savings test (verify >= 60% reduction) +7. Run `cargo fmt --all && cargo clippy --all-targets && cargo test` + +See the Filter Development Checklist in `CLAUDE.md` and `.claude/rules/rust-patterns.md` for the full module structure. diff --git a/src/cmds/cloud/README.md b/src/cmds/cloud/README.md new file mode 100644 index 00000000..5459b16e --- /dev/null +++ b/src/cmds/cloud/README.md @@ -0,0 +1,9 @@ +# Cloud and Infrastructure + +## Specifics + +- `aws_cmd.rs` forces `--output json` for structured parsing +- `container.rs` handles both Docker and Kubernetes; `DockerCommands` and `KubectlCommands` sub-enums in `main.rs` route to `container::run()` -- uses passthrough for unknown subcommands +- `curl_cmd.rs` auto-detects JSON responses and shows schema (structure without values) +- `wget_cmd.rs` wraps wget with output filtering +- `psql_cmd.rs` filters PostgreSQL query output diff --git a/src/cmds/dotnet/README.md b/src/cmds/dotnet/README.md new file mode 100644 index 00000000..f32eb384 --- /dev/null +++ b/src/cmds/dotnet/README.md @@ -0,0 +1,7 @@ +# .NET Ecosystem + +## Specifics + +- `dotnet_cmd.rs` uses `DotnetCommands` sub-enum in main.rs +- Internal helper modules (`dotnet_trx.rs`, `dotnet_format_report.rs`, `binlog.rs`) are only used by `dotnet_cmd.rs` -- they parse specialized .NET output formats (TRX XML, binary logs, format reports) +- Test fixtures are in `tests/fixtures/dotnet/` (JSON and text formats) diff --git a/src/cmds/git/README.md b/src/cmds/git/README.md new file mode 100644 index 00000000..bca9ef70 --- /dev/null +++ b/src/cmds/git/README.md @@ -0,0 +1,13 @@ +# Git and VCS + +## Specifics + +- **git.rs** uses `trailing_var_arg = true` + `allow_hyphen_values = true` so native git flags (`--oneline`, `--cached`, etc.) pass through correctly +- Auto-detects `--merges` flag to avoid conflicting with `--no-merges` injection +- Global git options (`-C`, `--git-dir`, `--work-tree`, `--no-pager`) are prepended before the subcommand +- Exit code propagation is critical for CI/CD pipelines + +## Cross-command + +- `gh_cmd.rs` imports markdown filtering helpers from `git.rs` for PR body rendering +- `diff_cmd.rs` is a standalone ultra-condensed diff (separate from `git diff`) diff --git a/src/cmds/go/README.md b/src/cmds/go/README.md new file mode 100644 index 00000000..95879efd --- /dev/null +++ b/src/cmds/go/README.md @@ -0,0 +1,7 @@ +# Go Ecosystem + +## Specifics + +- `go_cmd.rs` uses `GoCommands` sub-enum in main.rs (same pattern as git/cargo) +- `go test` outputs NDJSON (`-json` flag injected by RTK) -- parsed line-by-line as streaming events +- `golangci_cmd.rs` forces `--out-format=json` for structured parsing diff --git a/src/cmds/js/README.md b/src/cmds/js/README.md new file mode 100644 index 00000000..097a26d3 --- /dev/null +++ b/src/cmds/js/README.md @@ -0,0 +1,13 @@ +# JavaScript / TypeScript / Node + +## Specifics + +- `utils::package_manager_exec()` auto-detects pnpm/yarn/npm -- JS modules should use this instead of hardcoding a package manager +- `lint_cmd.rs` is a cross-ecosystem router: detects Python projects and delegates to `mypy_cmd` or `ruff_cmd` +- `vitest_cmd.rs` uses the `parser/` module for structured output parsing +- `playwright_cmd.rs` uses the `parser/` module for test result extraction + +## Cross-command + +- `lint_cmd` routes to `cmds/python/mypy_cmd` and `cmds/python/ruff_cmd` for Python projects +- `prettier_cmd` is also called by `cmds/system/format_cmd` as a format dispatcher target diff --git a/src/cmds/python/README.md b/src/cmds/python/README.md new file mode 100644 index 00000000..5330d66c --- /dev/null +++ b/src/cmds/python/README.md @@ -0,0 +1,13 @@ +# Python Ecosystem + +## Specifics + +- `pytest_cmd.rs` uses a state machine text parser (no JSON available from pytest) +- `ruff_cmd.rs` uses JSON for check mode (`--output-format=json`) and text filtering for format mode +- `pip_cmd.rs` auto-detects `uv` as a pip alternative and routes accordingly +- `python -m pytest` and `python3 -m mypy` are rewritten by the hook registry to `rtk pytest` / `rtk mypy` + +## Cross-command + +- `ruff_cmd` is called by `cmds/js/lint_cmd` and `cmds/system/format_cmd` for Python projects +- `mypy_cmd` is called by `cmds/js/lint_cmd` when detecting Python type checking diff --git a/src/cmds/ruby/README.md b/src/cmds/ruby/README.md new file mode 100644 index 00000000..0692458e --- /dev/null +++ b/src/cmds/ruby/README.md @@ -0,0 +1,9 @@ +# Ruby on Rails + +## Specifics + +- `rake_cmd.rs` filters Minitest output via `rake test` / `rails test`; state machine text parser, failures only (85-90% reduction) +- `rspec_cmd.rs` uses JSON injection (`--format json`) with text fallback; failures only (60%+ reduction) +- `rubocop_cmd.rs` uses JSON injection, groups by cop/severity (60%+ reduction) +- All three modules use `ruby_exec()` from `utils.rs` to auto-detect `bundle exec` when a Gemfile exists +- TOML filter `bundle-install.toml` strips `Using` lines from `bundle install`/`update` (90%+ reduction) diff --git a/src/cmds/rust/README.md b/src/cmds/rust/README.md new file mode 100644 index 00000000..42b6ac9d --- /dev/null +++ b/src/cmds/rust/README.md @@ -0,0 +1,7 @@ +# Rust Ecosystem + +## Specifics + +- `cargo_cmd.rs` uses `restore_double_dash()` fix: Clap strips `--` but cargo needs it for test flags (e.g., `cargo test -- --nocapture`) +- `runner.rs` is a generic two-mode runner (`err` = stderr only, `test` = failures only) used as fallback for commands without a dedicated filter +- `runner.rs` is also referenced by other modules outside this directory as a generic command executor diff --git a/src/cmds/system/README.md b/src/cmds/system/README.md new file mode 100644 index 00000000..fd503bdd --- /dev/null +++ b/src/cmds/system/README.md @@ -0,0 +1,12 @@ +# System and Generic Utilities + +## Specifics + +- `read.rs` uses `core/filter` for language-aware code stripping (FilterLevel: none/minimal/aggressive) +- `grep_cmd.rs` reads `core/config` for `limits.grep_max_results` and `limits.grep_max_per_file` +- `local_llm.rs` (`rtk smart`) uses `core/filter` for heuristic file summarization +- `format_cmd.rs` is a cross-ecosystem dispatcher: auto-detects and routes to `prettier_cmd`, `ruff_cmd`, or `black` + +## Cross-command + +- `format_cmd` routes to `cmds/js/prettier_cmd` and `cmds/python/ruff_cmd` diff --git a/src/core/README.md b/src/core/README.md new file mode 100644 index 00000000..2f0127f7 --- /dev/null +++ b/src/core/README.md @@ -0,0 +1,124 @@ +# Core Infrastructure + +## Scope + +Domain-agnostic building blocks with **no knowledge of any specific command, hook, or agent**. If a module references "git", "cargo", "claude", or any external tool by name, it does not belong here. Core is a leaf in the dependency graph — it is consumed by all other components but imports from none of them. + +Owns: configuration loading, token tracking persistence, TOML filter engine, tee output recovery, display formatting, telemetry, and shared utilities. + +Does **not** own: command-specific filtering logic (that's `cmds/`), hook lifecycle management (that's `src/hooks/`), or analytics dashboards (that's `analytics/`). + +## Purpose +Core infrastructure shared by all RTK command modules. These are the foundational building blocks that every filter, tracker, and command handler depends on. This module group has no inward dependencies -- it is a leaf in the dependency graph, ensuring clean layering across the codebase. + +## Files +| File | Responsibility | +|------|---------------| +| tracking.rs | SQLite-based persistent token metrics (1429 lines); records input/output token counts per command with project path; 90-day retention cleanup; token estimation via `ceil(chars / 4.0)` heuristic; `TimedExecution` API for timer-based tracking; imported by 46 files (highest dependency count); DB location: `RTK_DB_PATH` env > `config.toml` > `~/.local/share/rtk/tracking.db` | +| utils.rs | Shared utilities used across 41 files: `strip_ansi`, `truncate`, `execute_command`, `resolved_command`, `package_manager_exec`; package manager auto-detection (pnpm/yarn/npm/npx); consistent error handling and output formatting | +| config.rs | Configuration system reading `~/.config/rtk/config.toml`; sections: `[tracking]` (DB path, retention), `[display]` (colors, emoji, max_width), `[tee]` (recovery), `[telemetry]`, `[hooks]` (exclude_commands), `[limits]` (grep/status thresholds); on-demand loading (no startup I/O) | +| filter.rs | Language-aware code filtering engine with `FilterLevel` enum (`none`, `minimal`, `aggressive`); strips comments, whitespace, and function bodies based on level; supports Rust, Python, JS/TS, Java, Go, C/C++ via file extension detection with fallback heuristics | +| toml_filter.rs | TOML DSL filter engine (1686 lines); `TomlFilterRegistry` singleton via `lazy_static!`; three-tier lookup: project-local (trust-gated), user-global, built-in; 8-stage filter pipeline (see below); built-in filters compiled from `src/filters/*.toml` via `build.rs` | +| tee.rs | Raw output recovery on command failure (400 lines); saves unfiltered output to `~/.local/share/rtk/tee/{epoch}_{slug}.log`; prints one-line hint for LLM re-read; 20-file rotation, 1MB cap, min 500 chars; configurable via `[tee]` config section or `RTK_TEE`/`RTK_TEE_DIR` env vars; tee errors never affect command output or exit code | +| display_helpers.rs | Token display formatting helpers for consistent human-readable output of savings percentages, token counts, and comparison tables | +| telemetry.rs | Fire-and-forget usage telemetry (248 lines); non-blocking background thread; once per 23 hours via marker file; sends device hash (SHA-256 of hostname:username), version, OS, top commands, savings stats; 2-second timeout; disabled via `RTK_TELEMETRY_DISABLED=1` or `[telemetry] enabled = false` | + +## TOML Filter Pipeline + +The TOML DSL applies 8 stages in order: + +1. **strip_ansi**: Remove ANSI escape codes if enabled +2. **replace**: Line-by-line regex substitutions (chainable, supports backreferences) +3. **match_output**: Short-circuit rules (if output matches pattern, return message; `unless` field prevents swallowing errors) +4. **strip/keep_lines**: Filter lines by regex (mutually exclusive) +5. **truncate_lines_at**: Truncate each line to N chars (unicode-safe) +6. **head/tail_lines**: Keep first N or last N lines (with omit message) +7. **max_lines**: Absolute line cap applied after head/tail +8. **on_empty**: Return message if result is empty after all stages + +Three-tier filter lookup (first match wins): +1. `.rtk/filters.toml` (project-local, requires `rtk trust`) +2. `~/.config/rtk/filters.toml` (user-global) +3. Built-in filters concatenated by `build.rs` at compile time (57+ filters) + +## Tracking Database Schema + +```sql +CREATE TABLE commands ( + id INTEGER PRIMARY KEY, + timestamp TEXT, -- UTC ISO8601 + original_cmd TEXT, -- "ls -la" + rtk_cmd TEXT, -- "rtk ls" + project_path TEXT, -- cwd (for project-scoped stats) + input_tokens INTEGER, -- estimated from raw output + output_tokens INTEGER, -- estimated from filtered output + saved_tokens INTEGER, -- input - output + savings_pct REAL, -- (saved / input) * 100 + exec_time_ms INTEGER -- elapsed milliseconds +); + +CREATE TABLE parse_failures ( + id INTEGER PRIMARY KEY, + timestamp TEXT, + raw_command TEXT, + error_message TEXT, + fallback_succeeded INTEGER -- 1=yes, 0=no +); +``` + +Project-scoped queries use GLOB patterns (not LIKE) to avoid `_`/`%` wildcard issues in paths. + +## Config Sections + +```toml +[tracking] +enabled = true +history_days = 90 +database_path = "/custom/path/to/tracking.db" # Optional + +[display] +colors = true +emoji = true +max_width = 120 + +[tee] +enabled = true +mode = "failures" # failures | always | never +max_files = 20 +max_file_size = 1048576 +directory = "/custom/tee/dir" + +[telemetry] +enabled = true + +[hooks] +exclude_commands = ["curl", "playwright"] # Never auto-rewrite these + +[limits] +grep_max_results = 200 +grep_max_per_file = 25 +status_max_files = 15 +status_max_untracked = 10 +passthrough_max_chars = 2000 +``` + +## Consumer Contracts + +Core provides infrastructure that `cmds/` and other components consume. These contracts define expected usage. + +### Tracking (`TimedExecution`) + +Consumers must call `timer.track()` on **all** code paths — success, failure, and fallback. Calling `std::process::exit()` before `track()` loses metrics. The raw string passed to `track()` should include both stdout and stderr to produce accurate savings percentages. + +### Tee (`tee_and_hint`) + +Consumers that parse structured output (JSON, NDJSON, state machines) should call `tee::tee_and_hint()` to save raw output for LLM recovery on failure. Tee must be called before `std::process::exit()`. + +### Gaps (to be fixed) + +- `ls.rs`, `tree.rs` — exit before `track()` on error path (lost metrics) +- `container.rs` — inconsistent tracking across subcommands +- 26/38 command modules missing tee integration — see `src/cmds/README.md` for the full list + +## Adding New Functionality +Place new infrastructure code here if it meets **all** of these criteria: (1) it has no dependencies on command modules or hooks, (2) it is used by two or more other modules, and (3) it provides a general-purpose utility rather than command-specific logic. Follow the existing pattern of lazy-initialized resources (`lazy_static!` for regex, on-demand config loading) to preserve the <10ms startup target. Add `#[cfg(test)] mod tests` with unit tests in the same file. diff --git a/src/hooks/README.md b/src/hooks/README.md new file mode 100644 index 00000000..0e46ca94 --- /dev/null +++ b/src/hooks/README.md @@ -0,0 +1,83 @@ +# Hook System + +## Scope + +The **lifecycle management** layer for LLM agent hooks: install, uninstall, verify integrity, audit usage, and manage trust. This component creates and maintains the hook artifacts that live in `hooks/` (root), but does **not** execute rewrite logic itself — that lives in `discover/registry`. + +Owns: `rtk init` installation flows (4 agents via `AgentTarget` enum + 3 special modes: Gemini, Codex, OpenCode), SHA-256 integrity verification, hook version checking, audit log analysis, `rtk rewrite` CLI entry point, and TOML filter trust management. + +Does **not** own: the deployed hook scripts themselves (that's `hooks/`), the rewrite pattern registry (that's `discover/`), or command filtering (that's `cmds/`). + +Boundary notes: +- `rewrite_cmd.rs` is a thin CLI bridge — it exists to serve hooks (hooks call `rtk rewrite` as a subprocess) and delegates entirely to `discover/registry`. +- `trust.rs` gates project-local TOML filter execution. It lives here because the trust workflow is tied to hook-installed filter discovery, not to the core filter engine. + +## Purpose +LLM agent integration layer that installs, validates, and executes command-rewriting hooks for AI coding assistants. Hooks intercept raw CLI commands (e.g., `git status`) and rewrite them to RTK equivalents (e.g., `rtk git status`) so that LLM agents automatically benefit from token savings without explicit user configuration. + +## Files +| File | Responsibility | +|------|---------------| +| init.rs | `rtk init` command (2998 lines) -- orchestrates all installation/uninstallation flows for 4 agents (Claude, Cursor, Windsurf, Cline) + 3 special modes (Gemini, Codex, OpenCode); supports 6 installation modes (default, hook-only, claude-md, windsurf, cline, codex); handles settings.json patching, RTK.md writing, CLAUDE.md injection, and OpenCode/Cursor side-installs | +| hook_cmd.rs | Hook processors for Gemini CLI (`run_gemini()`) and GitHub Copilot (`run_copilot()`); reads JSON from stdin, auto-detects agent format (VS Code vs Copilot CLI), calls `rewrite_command()` in-process, returns agent-specific JSON response | +| hook_check.rs | Runtime hook version detection; parses `# rtk-hook-version: N` from hook script header; warns if outdated or missing; rate-limited to once per 24 hours via marker file at `~/.local/share/rtk/.hook_warn_last` | +| hook_audit_cmd.rs | `rtk hook-audit` command; analyzes hook audit log (`~/.local/share/rtk/hook-audit.log`) enabled via `RTK_HOOK_AUDIT=1`; shows rewrite success rates, skip reasons, and top commands | +| rewrite_cmd.rs | `rtk rewrite` command -- thin wrapper that loads config exclusions and delegates to `discover/registry::rewrite_command()`; used by all shell-based hooks as a subprocess | +| verify_cmd.rs | `rtk verify` command -- runs inline tests from TOML filter files; integrity verification (`integrity::run_verify()`) is routed via `main.rs`, not this module | +| trust.rs | `rtk trust` / `rtk untrust` commands -- manages a trust store for project-local TOML filters in `.rtk/filters/`; prevents untrusted filters from executing | +| integrity.rs | SHA-256 hook integrity system (538 lines); computes and stores hashes at install time; verifies at runtime; 5-state model: Verified, Tampered, NoBaseline, NotInstalled, OrphanedHash | + +## Installation Modes + +`rtk init` supports 6 distinct installation flows: + +| Mode | Command | Creates | Patches | +|------|---------|---------|---------| +| Default (global) | `rtk init -g` | Hook, SHA-256 hash, RTK.md | settings.json, CLAUDE.md | +| Hook only | `rtk init -g --hook-only` | Hook, SHA-256 hash | settings.json | +| Claude-MD (legacy) | `rtk init --claude-md` | 134-line RTK block | CLAUDE.md | +| Windsurf | `rtk init -g --agent windsurf` | `.windsurfrules` | -- | +| Cline | `rtk init --agent cline` | `.clinerules` | -- | +| Codex | `rtk init --codex` | RTK.md | AGENTS.md | +| Cursor | `rtk init -g --agent cursor` | Cursor hook | hooks.json | + + +## Integrity Verification + +The integrity system prevents unauthorized hook modifications: + +1. At install: `integrity::store_hash()` computes SHA-256 of the hook file, writes to `~/.claude/hooks/.rtk-hook.sha256` (read-only 0o444) +2. At runtime: `integrity::runtime_check()` re-computes hash and compares; blocks execution if tampered +3. On demand: `rtk verify` prints detailed verification status (PASS/FAIL/WARN/SKIP) + +Five integrity states: +- **Verified**: Hash matches stored value +- **Tampered**: Hash mismatch (blocks execution) +- **NoBaseline**: Hook exists but no hash stored (old install) +- **NotInstalled**: No hook, no hash +- **OrphanedHash**: Hash file exists, hook missing + +## PatchMode Behavior + +Controls how `rtk init` modifies agent settings files: + +| Mode | Flag | Behavior | +|------|------|----------| +| Ask (default) | -- | Prompts user `[y/N]`; defaults to No if stdin not terminal | +| Auto | `--auto-patch` | Patches without prompting; for CI/scripted installs | +| Skip | `--no-patch` | Prints manual instructions; user patches manually | + +## Atomicity and Safety + +All file operations use atomic writes (tempfile + rename) to prevent corruption on crash. Settings files are backed up to `.bak` before modification. All operations are idempotent -- running `rtk init` multiple times is safe. + +## Exit Code Contract + +Hook processors in `hook_cmd.rs` must return `Ok(())` on every path — success, no-match, parse error, and unexpected input. Returning `Err` propagates to `main()` and exits non-zero, which blocks the agent's command from executing. This violates the non-blocking guarantee documented in `hooks/README.md`. + +### Gaps (to be fixed) + +- `hook_cmd.rs::run_gemini()` — uses `.context()?` on JSON parse, which returns `Err` on malformed input + +## Adding New Functionality +To add support for a new AI coding agent: (1) add the hook installation logic to `init.rs` following the existing agent patterns, (2) if the agent requires a custom hook protocol (like Gemini's `BeforeTool`), add a processor function in `hook_cmd.rs`, (3) add the agent's hook file path to `hook_check.rs` for validation, and (4) update `integrity.rs` with the expected hash for the new hook file. Test by running `rtk init` in a fresh environment and verifying the hook rewrites commands correctly in the target agent. From 46fa31c4b8a60f8f9b1e767b87dba6f54dd9e901 Mon Sep 17 00:00:00 2001 From: aesoft <43991222+aeppling@users.noreply.github.com> Date: Tue, 24 Mar 2026 22:14:35 +0100 Subject: [PATCH 2/5] chore(refacto-codebase): Folders + Technical docs - codebase more clear for humans and AI agents - alignement on vision and filter quality in technical docs --- .claude/agents/code-reviewer.md | 16 +- .claude/agents/rtk-testing-specialist.md | 31 +- .claude/agents/rust-rtk.md | 62 +- .claude/agents/system-architect.md | 27 +- .claude/agents/technical-writer.md | 4 +- .claude/commands/tech/codereview.md | 9 +- .claude/rules/cli-testing.md | 30 +- .claude/rules/search-strategy.md | 75 ++- .github/copilot-instructions.md | 101 +--- ARCHITECTURE.md | 551 ++---------------- CLAUDE.md | 532 ++--------------- CONTRIBUTING.md | 86 ++- docs/TECHNICAL.md | 66 +-- hooks/README.md | 44 +- hooks/claude/README.md | 2 + hooks/{ => claude}/rtk-awareness.md | 0 hooks/{ => claude}/rtk-rewrite.sh | 0 hooks/{ => claude}/test-rtk-rewrite.sh | 0 hooks/cline/README.md | 2 + hooks/{cline-rtk-rules.md => cline/rules.md} | 0 hooks/codex/README.md | 2 + .../rtk-awareness.md} | 0 hooks/copilot/README.md | 2 + .../rtk-awareness.md} | 0 .../test-rtk-rewrite.sh} | 0 hooks/cursor/README.md | 2 + .../rtk-rewrite.sh} | 0 hooks/opencode/README.md | 2 + hooks/{opencode-rtk.ts => opencode/rtk.ts} | 0 hooks/windsurf/README.md | 2 + .../rules.md} | 0 src/analytics/README.md | 12 +- src/{ => analytics}/cc_economics.rs | 12 +- src/{ => analytics}/ccusage.rs | 4 +- src/{ => analytics}/gain.rs | 10 +- src/analytics/mod.rs | 6 + src/{ => analytics}/session_cmd.rs | 4 +- src/cmds/README.md | 80 +-- src/cmds/cloud/README.md | 2 + src/{ => cmds/cloud}/aws_cmd.rs | 4 +- src/{ => cmds/cloud}/container.rs | 6 +- src/{ => cmds/cloud}/curl_cmd.rs | 6 +- src/cmds/cloud/mod.rs | 7 + src/{ => cmds/cloud}/psql_cmd.rs | 6 +- src/{ => cmds/cloud}/wget_cmd.rs | 4 +- src/cmds/dotnet/README.md | 2 + src/{ => cmds/dotnet}/binlog.rs | 8 +- src/{ => cmds/dotnet}/dotnet_cmd.rs | 6 +- src/{ => cmds/dotnet}/dotnet_format_report.rs | 2 + src/{ => cmds/dotnet}/dotnet_trx.rs | 2 + src/cmds/dotnet/mod.rs | 6 + src/cmds/git/README.md | 2 + src/{ => cmds/git}/diff_cmd.rs | 6 +- src/{ => cmds/git}/gh_cmd.rs | 4 +- src/{ => cmds/git}/git.rs | 8 +- src/{ => cmds/git}/gt_cmd.rs | 8 +- src/cmds/git/mod.rs | 6 + src/cmds/go/README.md | 2 + src/{ => cmds/go}/go_cmd.rs | 12 +- src/{ => cmds/go}/golangci_cmd.rs | 10 +- src/cmds/go/mod.rs | 4 + src/cmds/js/README.md | 2 + src/{ => cmds/js}/lint_cmd.rs | 10 +- src/cmds/js/mod.rs | 11 + src/{ => cmds/js}/next_cmd.rs | 6 +- src/{ => cmds/js}/npm_cmd.rs | 6 +- src/{ => cmds/js}/playwright_cmd.rs | 8 +- src/{ => cmds/js}/pnpm_cmd.rs | 6 +- src/{ => cmds/js}/prettier_cmd.rs | 6 +- src/{ => cmds/js}/prisma_cmd.rs | 6 +- src/{ => cmds/js}/tsc_cmd.rs | 8 +- src/{ => cmds/js}/vitest_cmd.rs | 8 +- src/cmds/mod.rs | 11 + src/cmds/python/README.md | 2 + src/cmds/python/mod.rs | 6 + src/{ => cmds/python}/mypy_cmd.rs | 6 +- src/{ => cmds/python}/pip_cmd.rs | 6 +- src/{ => cmds/python}/pytest_cmd.rs | 8 +- src/{ => cmds/python}/ruff_cmd.rs | 8 +- src/cmds/ruby/README.md | 2 + src/cmds/ruby/mod.rs | 5 + src/{ => cmds/ruby}/rake_cmd.rs | 10 +- src/{ => cmds/ruby}/rspec_cmd.rs | 8 +- src/{ => cmds/ruby}/rubocop_cmd.rs | 12 +- src/cmds/rust/README.md | 2 + src/{ => cmds/rust}/cargo_cmd.rs | 8 +- src/cmds/rust/mod.rs | 4 + src/{ => cmds/rust}/runner.rs | 8 +- src/cmds/system/README.md | 2 + src/{ => cmds/system}/deps.rs | 4 +- src/{ => cmds/system}/env_cmd.rs | 4 +- src/{ => cmds/system}/find_cmd.rs | 4 +- src/{ => cmds/system}/format_cmd.rs | 6 +- src/{ => cmds/system}/grep_cmd.rs | 8 +- src/{ => cmds/system}/json_cmd.rs | 4 +- src/{ => cmds/system}/local_llm.rs | 4 +- src/{ => cmds/system}/log_cmd.rs | 4 +- src/{ => cmds/system}/ls.rs | 6 +- src/cmds/system/mod.rs | 15 + src/{ => cmds/system}/read.rs | 6 +- src/{ => cmds/system}/summary.rs | 6 +- src/{ => cmds/system}/tree.rs | 4 +- src/{ => cmds/system}/wc_cmd.rs | 4 +- src/core/README.md | 31 +- src/{ => core}/config.rs | 4 +- src/{ => core}/display_helpers.rs | 6 +- src/{ => core}/filter.rs | 2 + src/core/mod.rs | 10 + src/{ => core}/tee.rs | 4 +- src/{ => core}/telemetry.rs | 6 +- src/{ => core}/toml_filter.rs | 20 +- src/{ => core}/tracking.rs | 2 +- src/{ => core}/utils.rs | 0 src/discover/README.md | 32 + src/discover/mod.rs | 2 + src/discover/provider.rs | 2 + src/discover/registry.rs | 2 + src/discover/report.rs | 2 + src/discover/rules.rs | 2 + src/filters/README.md | 14 + src/hooks/README.md | 14 +- src/{ => hooks}/hook_audit_cmd.rs | 2 + src/{ => hooks}/hook_check.rs | 2 + src/{ => hooks}/hook_cmd.rs | 4 +- src/{ => hooks}/init.rs | 20 +- src/{ => hooks}/integrity.rs | 2 +- src/hooks/mod.rs | 10 + src/{ => hooks}/rewrite_cmd.rs | 4 +- src/{ => hooks}/trust.rs | 4 +- src/{ => hooks}/verify_cmd.rs | 4 +- src/learn/README.md | 27 + src/learn/detector.rs | 2 + src/learn/mod.rs | 2 + src/learn/report.rs | 2 + src/main.rs | 177 +++--- src/parser/README.md | 148 +---- src/parser/mod.rs | 2 +- 137 files changed, 1012 insertions(+), 1718 deletions(-) rename hooks/{ => claude}/rtk-awareness.md (100%) rename hooks/{ => claude}/rtk-rewrite.sh (100%) rename hooks/{ => claude}/test-rtk-rewrite.sh (100%) mode change 100755 => 100644 rename hooks/{cline-rtk-rules.md => cline/rules.md} (100%) rename hooks/{rtk-awareness-codex.md => codex/rtk-awareness.md} (100%) rename hooks/{copilot-rtk-awareness.md => copilot/rtk-awareness.md} (100%) rename hooks/{test-copilot-rtk-rewrite.sh => copilot/test-rtk-rewrite.sh} (100%) mode change 100755 => 100644 rename hooks/{cursor-rtk-rewrite.sh => cursor/rtk-rewrite.sh} (100%) mode change 100755 => 100644 rename hooks/{opencode-rtk.ts => opencode/rtk.ts} (100%) rename hooks/{windsurf-rtk-rules.md => windsurf/rules.md} (100%) rename src/{ => analytics}/cc_economics.rs (99%) rename src/{ => analytics}/ccusage.rs (98%) rename src/{ => analytics}/gain.rs (98%) create mode 100644 src/analytics/mod.rs rename src/{ => analytics}/session_cmd.rs (99%) rename src/{ => cmds/cloud}/aws_cmd.rs (99%) rename src/{ => cmds/cloud}/container.rs (99%) rename src/{ => cmds/cloud}/curl_cmd.rs (96%) create mode 100644 src/cmds/cloud/mod.rs rename src/{ => cmds/cloud}/psql_cmd.rs (98%) rename src/{ => cmds/cloud}/wget_cmd.rs (99%) rename src/{ => cmds/dotnet}/binlog.rs (99%) rename src/{ => cmds/dotnet}/dotnet_cmd.rs (99%) rename src/{ => cmds/dotnet}/dotnet_format_report.rs (98%) rename src/{ => cmds/dotnet}/dotnet_trx.rs (99%) create mode 100644 src/cmds/dotnet/mod.rs rename src/{ => cmds/git}/diff_cmd.rs (98%) rename src/{ => cmds/git}/gh_cmd.rs (99%) rename src/{ => cmds/git}/git.rs (99%) rename src/{ => cmds/git}/gt_cmd.rs (98%) create mode 100644 src/cmds/git/mod.rs rename src/{ => cmds/go}/go_cmd.rs (97%) rename src/{ => cmds/go}/golangci_cmd.rs (98%) create mode 100644 src/cmds/go/mod.rs rename src/{ => cmds/js}/lint_cmd.rs (98%) create mode 100644 src/cmds/js/mod.rs rename src/{ => cmds/js}/next_cmd.rs (97%) rename src/{ => cmds/js}/npm_cmd.rs (97%) rename src/{ => cmds/js}/playwright_cmd.rs (98%) rename src/{ => cmds/js}/pnpm_cmd.rs (99%) rename src/{ => cmds/js}/prettier_cmd.rs (97%) rename src/{ => cmds/js}/prisma_cmd.rs (98%) rename src/{ => cmds/js}/tsc_cmd.rs (97%) rename src/{ => cmds/js}/vitest_cmd.rs (98%) create mode 100644 src/cmds/mod.rs create mode 100644 src/cmds/python/mod.rs rename src/{ => cmds/python}/mypy_cmd.rs (98%) rename src/{ => cmds/python}/pip_cmd.rs (98%) rename src/{ => cmds/python}/pytest_cmd.rs (97%) rename src/{ => cmds/python}/ruff_cmd.rs (98%) create mode 100644 src/cmds/ruby/mod.rs rename src/{ => cmds/ruby}/rake_cmd.rs (98%) rename src/{ => cmds/ruby}/rspec_cmd.rs (99%) rename src/{ => cmds/ruby}/rubocop_cmd.rs (98%) rename src/{ => cmds/rust}/cargo_cmd.rs (99%) create mode 100644 src/cmds/rust/mod.rs rename src/{ => cmds/rust}/runner.rs (96%) rename src/{ => cmds/system}/deps.rs (98%) rename src/{ => cmds/system}/env_cmd.rs (98%) rename src/{ => cmds/system}/find_cmd.rs (99%) rename src/{ => cmds/system}/format_cmd.rs (98%) rename src/{ => cmds/system}/grep_cmd.rs (98%) rename src/{ => cmds/system}/json_cmd.rs (99%) rename src/{ => cmds/system}/local_llm.rs (98%) rename src/{ => cmds/system}/log_cmd.rs (98%) rename src/{ => cmds/system}/ls.rs (98%) create mode 100644 src/cmds/system/mod.rs rename src/{ => cmds/system}/read.rs (97%) rename src/{ => cmds/system}/summary.rs (98%) rename src/{ => cmds/system}/tree.rs (98%) rename src/{ => cmds/system}/wc_cmd.rs (99%) rename src/{ => core}/config.rs (98%) rename src/{ => core}/display_helpers.rs (98%) rename src/{ => core}/filter.rs (99%) create mode 100644 src/core/mod.rs rename src/{ => core}/tee.rs (99%) rename src/{ => core}/telemetry.rs (98%) rename src/{ => core}/toml_filter.rs (98%) rename src/{ => core}/tracking.rs (99%) rename src/{ => core}/utils.rs (100%) create mode 100644 src/discover/README.md rename src/{ => hooks}/hook_audit_cmd.rs (99%) rename src/{ => hooks}/hook_check.rs (98%) rename src/{ => hooks}/hook_cmd.rs (98%) rename src/{ => hooks}/init.rs (99%) rename src/{ => hooks}/integrity.rs (99%) create mode 100644 src/hooks/mod.rs rename src/{ => hooks}/rewrite_cmd.rs (90%) rename src/{ => hooks}/trust.rs (99%) rename src/{ => hooks}/verify_cmd.rs (92%) create mode 100644 src/learn/README.md diff --git a/.claude/agents/code-reviewer.md b/.claude/agents/code-reviewer.md index 8702a5db..f746c603 100644 --- a/.claude/agents/code-reviewer.md +++ b/.claude/agents/code-reviewer.md @@ -14,13 +14,15 @@ Prevent bugs, performance regressions, and token savings failures before they re ## RTK Architecture Context ``` -main.rs (Commands enum + routing) - → *_cmd.rs modules (filter logic) - → tracking.rs (SQLite, token metrics) - → utils.rs (shared helpers) - → tee.rs (failure recovery) - → config.rs (user config) - → filter.rs (language-aware filtering) +src/main.rs (Commands enum + routing) + → src/cmds/**/*_cmd.rs (filter logic, organized by ecosystem) + → src/core/tracking.rs (SQLite, token metrics) + → src/core/utils.rs (shared helpers) + → src/core/tee.rs (failure recovery) + → src/core/config.rs (user config) + → src/core/filter.rs (language-aware filtering) + → src/hooks/ (init, rewrite, verify, trust) + → src/analytics/ (gain, cc_economics, ccusage) ``` **Non-negotiable constraints:** diff --git a/.claude/agents/rtk-testing-specialist.md b/.claude/agents/rtk-testing-specialist.md index 54767b70..649414e3 100644 --- a/.claude/agents/rtk-testing-specialist.md +++ b/.claude/agents/rtk-testing-specialist.md @@ -337,7 +337,7 @@ docker run --rm -v $(pwd):/rtk -w /rtk rust:latest cargo test newcmd --some-args > tests/fixtures/newcmd_raw.txt ``` -2. **Add snapshot test** to `src/newcmd_cmd.rs`: +2. **Add snapshot test** to `src/cmds//newcmd_cmd.rs`: ```rust #[cfg(test)] mod tests { @@ -439,18 +439,27 @@ cargo test --ignored ``` rtk/ ├── src/ -│ ├── git.rs # Filter implementation -│ │ └── #[cfg(test)] mod tests { ... } # Unit tests -│ ├── snapshots/ # Insta snapshots (gitignored pattern) -│ │ └── git.rs.snap # Snapshot for git tests +│ ├── cmds/ +│ │ ├── git/ +│ │ │ ├── git.rs # Filter implementation +│ │ │ │ └── #[cfg(test)] mod tests { ... } # Unit tests +│ │ │ └── snapshots/ # Insta snapshots for git module +│ │ ├── js/ +│ │ ├── python/ +│ │ └── ... # Other ecosystems +│ ├── core/ +│ │ ├── filter.rs # Core filtering with tests +│ │ └── snapshots/ +│ └── hooks/ ├── tests/ │ ├── common/ -│ │ └── mod.rs # Shared test utilities (count_tokens, etc.) -│ ├── fixtures/ # Real command output fixtures -│ │ ├── git_log_raw.txt # Real git log output -│ │ ├── cargo_test_raw.txt # Real cargo test output -│ │ └── gh_pr_view_raw.txt # Real gh pr view output -│ └── integration_test.rs # Integration tests (#[ignore]) +│ │ └── mod.rs # Shared test utilities (count_tokens, etc.) +│ ├── fixtures/ # Real command output fixtures +│ │ ├── git_log_raw.txt +│ │ ├── cargo_test_raw.txt +│ │ ├── gh_pr_view_raw.txt +│ │ └── dotnet/ # Dotnet-specific fixtures +│ └── integration_test.rs # Integration tests (#[ignore]) ``` **Best practices**: diff --git a/.claude/agents/rust-rtk.md b/.claude/agents/rust-rtk.md index 600c2f91..8efe67f0 100644 --- a/.claude/agents/rust-rtk.md +++ b/.claude/agents/rust-rtk.md @@ -306,20 +306,27 @@ fn test_real_git_log() { ## Key Files Reference -**Core modules**: +**Core infrastructure** (`src/core/`): - `src/main.rs` - CLI entry point, Clap command parsing, routing to modules -- `src/git.rs` - Git operations filter (log, status, diff, etc.) -- `src/grep_cmd.rs` - Code search filter (grep, ripgrep) -- `src/runner.rs` - Command execution filter (test, err) -- `src/utils.rs` - Shared utilities (truncate, strip_ansi, execute_command) -- `src/tracking.rs` - SQLite token savings tracking (`rtk gain`) - -**Filter modules** (see CLAUDE.md Module Responsibilities table): -- `src/lint_cmd.rs`, `src/tsc_cmd.rs`, `src/next_cmd.rs` - JavaScript/TypeScript tooling -- `src/prettier_cmd.rs`, `src/playwright_cmd.rs`, `src/prisma_cmd.rs` - Modern JS stack -- `src/pnpm_cmd.rs`, `src/vitest_cmd.rs` - Package manager, test runner -- `src/ruff_cmd.rs`, `src/pytest_cmd.rs`, `src/pip_cmd.rs` - Python ecosystem -- `src/go_cmd.rs`, `src/golangci_cmd.rs` - Go ecosystem +- `src/core/utils.rs` - Shared utilities (truncate, strip_ansi, execute_command) +- `src/core/tracking.rs` - SQLite token savings tracking (`rtk gain`) +- `src/core/filter.rs` - Language-aware code filtering engine +- `src/core/tee.rs` - Raw output recovery on failure +- `src/core/config.rs` - User configuration (~/.config/rtk/config.toml) + +**Command modules** (`src/cmds//`): +- `src/cmds/git/` - git.rs, gh_cmd.rs, gt_cmd.rs, diff_cmd.rs +- `src/cmds/rust/` - cargo_cmd.rs, runner.rs +- `src/cmds/js/` - lint_cmd.rs, tsc_cmd.rs, next_cmd.rs, prettier_cmd.rs, playwright_cmd.rs, prisma_cmd.rs, vitest_cmd.rs, pnpm_cmd.rs, npm_cmd.rs +- `src/cmds/python/` - ruff_cmd.rs, pytest_cmd.rs, mypy_cmd.rs, pip_cmd.rs +- `src/cmds/go/` - go_cmd.rs, golangci_cmd.rs +- `src/cmds/ruby/` - rake_cmd.rs, rspec_cmd.rs, rubocop_cmd.rs +- `src/cmds/cloud/` - aws_cmd.rs, container.rs, curl_cmd.rs, wget_cmd.rs, psql_cmd.rs +- `src/cmds/system/` - ls.rs, tree.rs, read.rs, grep_cmd.rs, find_cmd.rs, etc. + +**Hook & analytics** (`src/hooks/`, `src/analytics/`): +- `src/hooks/init.rs` - rtk init command +- `src/analytics/gain.rs` - rtk gain command **Tests**: - `tests/fixtures/` - Real command output fixtures for testing @@ -401,11 +408,11 @@ When adding a new filter (e.g., `rtk newcmd`): ### 1. Create Module ```bash -touch src/newcmd_cmd.rs +touch src/cmds//newcmd_cmd.rs ``` ```rust -// src/newcmd_cmd.rs +// src/cmds//newcmd_cmd.rs use anyhow::{Context, Result}; use lazy_static::lazy_static; use regex::Regex; @@ -436,18 +443,23 @@ mod tests { } ``` -### 2. Add to main.rs Commands Enum +### 2. Register Module +Add to ecosystem `mod.rs` (e.g., `src/cmds/system/mod.rs`): ```rust -// src/main.rs -#[derive(Subcommand)] -enum Commands { - // ... existing commands - Newcmd { - #[arg(trailing_var_arg = true, allow_hyphen_values = true)] - args: Vec, - }, -} +pub mod newcmd_cmd; +``` + +Add to `src/main.rs` Commands enum and routing: +```rust +// Add use import +use cmds::system::newcmd_cmd; + +// In Commands enum +Newcmd { + #[arg(trailing_var_arg = true, allow_hyphen_values = true)] + args: Vec, +}, // In match statement Commands::Newcmd { args } => { diff --git a/.claude/agents/system-architect.md b/.claude/agents/system-architect.md index 6de564e7..790deb4a 100644 --- a/.claude/agents/system-architect.md +++ b/.claude/agents/system-architect.md @@ -30,21 +30,24 @@ Think in terms of filter families, not individual commands. Every new `*_cmd.rs` ## RTK Architecture Map ``` -main.rs +src/main.rs ├── Commands enum (clap derive) -│ ├── Git(GitArgs) → git.rs -│ ├── Cargo(CargoArgs) → runner.rs -│ ├── Gh(GhArgs) → gh_cmd.rs -│ ├── Grep(GrepArgs) → grep_cmd.rs -│ ├── ... → *_cmd.rs -│ ├── Gain → tracking.rs +│ ├── Git(GitArgs) → cmds/git/git.rs +│ ├── Cargo(CargoArgs) → cmds/rust/runner.rs +│ ├── Gh(GhArgs) → cmds/git/gh_cmd.rs +│ ├── Grep(GrepArgs) → cmds/system/grep_cmd.rs +│ ├── ... → cmds//*_cmd.rs +│ ├── Gain → analytics/gain.rs │ └── Proxy(ProxyArgs) → passthrough │ -├── tracking.rs ← SQLite, token metrics, 90-day retention -├── config.rs ← ~/.config/rtk/config.toml -├── tee.rs ← Raw output recovery on failure -├── filter.rs ← Language-aware code filtering -└── utils.rs ← strip_ansi, truncate, execute_command +├── core/ +│ ├── tracking.rs ← SQLite, token metrics, 90-day retention +│ ├── config.rs ← ~/.config/rtk/config.toml +│ ├── tee.rs ← Raw output recovery on failure +│ ├── filter.rs ← Language-aware code filtering +│ └── utils.rs ← strip_ansi, truncate, execute_command +├── hooks/ ← init, rewrite, verify, trust, integrity +└── analytics/ ← gain, cc_economics, ccusage, session_cmd ``` **TOML Filter DSL** (v0.25.0+): diff --git a/.claude/agents/technical-writer.md b/.claude/agents/technical-writer.md index 9f41e3e9..f5341af4 100644 --- a/.claude/agents/technical-writer.md +++ b/.claude/agents/technical-writer.md @@ -227,11 +227,11 @@ echo $LAST_COMMAND # Should show "rtk git status" ### 1. Create Filter Module ```bash -touch src/newcmd_cmd.rs +touch src/cmds//newcmd_cmd.rs ``` ```rust -// src/newcmd_cmd.rs +// src/cmds//newcmd_cmd.rs use anyhow::{Context, Result}; use lazy_static::lazy_static; use regex::Regex; diff --git a/.claude/commands/tech/codereview.md b/.claude/commands/tech/codereview.md index 46b3579d..fb0813fc 100644 --- a/.claude/commands/tech/codereview.md +++ b/.claude/commands/tech/codereview.md @@ -57,11 +57,12 @@ git diff "$BASE_BRANCH"...HEAD --stat | Si le diff contient... | Vérifier | | ------------------------------ | ------------------------------------------ | -| `src/*.rs` | CLAUDE.md sections Error Handling + Tests | -| `src/filter.rs` ou `*_cmd.rs` | Filter Development Checklist (CLAUDE.md) | +| `src/**/*.rs` | CLAUDE.md sections Error Handling + Tests | +| `src/core/filter.rs` ou `src/cmds/**/*_cmd.rs` | Filter Development Checklist (CLAUDE.md) | | `src/main.rs` | Command routing + Commands enum | -| `src/tracking.rs` | SQLite patterns + DB path config | -| `src/config.rs` | Configuration system + init patterns | +| `src/core/tracking.rs` | SQLite patterns + DB path config | +| `src/core/config.rs` | Configuration system | +| `src/hooks/init.rs` | Init patterns + hook installation | | `.github/workflows/` | CI/CD multi-platform build targets | | `tests/` ou `fixtures/` | Testing Strategy (CLAUDE.md) | | `Cargo.toml` | Dependencies + build optimizations | diff --git a/.claude/rules/cli-testing.md b/.claude/rules/cli-testing.md index 7e04c4c8..58fdad60 100644 --- a/.claude/rules/cli-testing.md +++ b/.claude/rules/cli-testing.md @@ -43,7 +43,7 @@ fn test_git_log_output() { git log -20 > tests/fixtures/git_log_raw.txt # 2. Write test with assert_snapshot! -cat > src/git.rs <<'EOF' +cat > src/cmds/git/git.rs <<'EOF' #[cfg(test)] mod tests { use insta::assert_snapshot; @@ -64,7 +64,7 @@ cargo test test_git_log_format cargo insta review # Press 'a' to accept, 'r' to reject -# 5. Snapshot saved in src/snapshots/git.rs.snap +# 5. Snapshot saved in src/cmds/git/snapshots/git__tests__*.snap ``` ## Token Accuracy Testing (🔴 Critical) @@ -299,24 +299,32 @@ diff /tmp/before.txt /tmp/after.txt ``` rtk/ ├── src/ -│ ├── git.rs # Filter implementation -│ │ └── #[cfg(test)] mod tests { ... } # Unit tests -│ ├── snapshots/ # Insta snapshots -│ │ └── git.rs.snap # Snapshot for git tests +│ ├── cmds/ +│ │ ├── git/ +│ │ │ ├── git.rs # Filter implementation +│ │ │ │ └── #[cfg(test)] mod tests { ... } +│ │ │ └── snapshots/ # Insta snapshots for git module +│ │ ├── js/ # JS/TS ecosystem filters +│ │ ├── python/ # Python ecosystem filters +│ │ └── ... +│ ├── core/ # Shared infrastructure +│ ├── hooks/ # Hook system +│ └── analytics/ # Token savings analytics ├── tests/ │ ├── common/ -│ │ └── mod.rs # Shared test utilities (count_tokens) -│ ├── fixtures/ # Real command output +│ │ └── mod.rs # Shared test utilities (count_tokens) +│ ├── fixtures/ # Real command output │ │ ├── git_log_raw.txt │ │ ├── cargo_test_raw.txt -│ │ └── gh_pr_view_raw.txt -│ └── integration_test.rs # Integration tests (#[ignore]) +│ │ ├── gh_pr_view_raw.txt +│ │ └── dotnet/ # Dotnet-specific fixtures +│ └── integration_test.rs # Integration tests (#[ignore]) ``` **Best practices**: - **Unit tests**: Embedded in module (`#[cfg(test)] mod tests`) - **Fixtures**: Real command output in `tests/fixtures/` -- **Snapshots**: Auto-generated in `src/snapshots/` (by insta) +- **Snapshots**: Auto-generated in `src/cmds//snapshots/` (by insta) - **Shared utils**: `tests/common/mod.rs` (count_tokens, helpers) - **Integration**: `tests/` with `#[ignore]` attribute diff --git a/.claude/rules/search-strategy.md b/.claude/rules/search-strategy.md index c12aef7b..0b4504df 100644 --- a/.claude/rules/search-strategy.md +++ b/.claude/rules/search-strategy.md @@ -15,20 +15,43 @@ Never use Bash for search (`find`, `grep`, `rg`) — use dedicated tools. ``` src/ -├── main.rs ← Commands enum + routing (start here for any command) -├── git.rs ← Git operations (log, status, diff) -├── runner.rs ← Cargo commands (test, build, clippy, check) -├── gh_cmd.rs ← GitHub CLI (pr, run, issue) -├── grep_cmd.rs ← Code search output filtering -├── ls.rs ← Directory listing -├── read.rs ← File reading with filter levels -├── filter.rs ← Language-aware code filtering engine -├── tracking.rs ← SQLite token metrics -├── config.rs ← ~/.config/rtk/config.toml -├── tee.rs ← Raw output recovery on failure -├── utils.rs ← strip_ansi, truncate, execute_command -├── init.rs ← rtk init command -└── *_cmd.rs ← All other command modules +├── main.rs ← Commands enum + routing (start here for any command) +├── core/ ← Shared infrastructure +│ ├── config.rs ← ~/.config/rtk/config.toml +│ ├── tracking.rs ← SQLite token metrics +│ ├── tee.rs ← Raw output recovery on failure +│ ├── utils.rs ← strip_ansi, truncate, execute_command +│ ├── filter.rs ← Language-aware code filtering engine +│ ├── toml_filter.rs ← TOML DSL filter engine +│ ├── display_helpers.rs ← Terminal formatting helpers +│ └── telemetry.rs ← Analytics ping +├── hooks/ ← Hook system +│ ├── init.rs ← rtk init command +│ ├── rewrite_cmd.rs ← rtk rewrite command +│ ├── hook_cmd.rs ← Gemini/Copilot hook processors +│ ├── hook_check.rs ← Hook status detection +│ ├── verify_cmd.rs ← rtk verify command +│ ├── trust.rs ← Project trust/untrust +│ └── integrity.rs ← SHA-256 hook verification +├── analytics/ ← Token savings analytics +│ ├── gain.rs ← rtk gain command +│ ├── cc_economics.rs ← Claude Code economics +│ ├── ccusage.rs ← ccusage data parsing +│ └── session_cmd.rs ← Session adoption reporting +├── cmds/ ← Command filter modules +│ ├── git/ ← git, gh, gt, diff +│ ├── rust/ ← cargo, runner (err/test) +│ ├── js/ ← npm, pnpm, vitest, lint, tsc, next, prettier, playwright, prisma +│ ├── python/ ← ruff, pytest, mypy, pip +│ ├── go/ ← go, golangci-lint +│ ├── dotnet/ ← dotnet, binlog, trx, format_report +│ ├── cloud/ ← aws, container (docker/kubectl), curl, wget, psql +│ ├── system/ ← ls, tree, read, grep, find, wc, env, json, log, deps, summary, format, local_llm +│ └── ruby/ ← rake, rspec, rubocop +├── discover/ ← Claude Code history analysis +├── learn/ ← CLI correction detection +├── parser/ ← Parser infrastructure +└── filters/ ← 60 TOML filter configs ``` ## Common Search Patterns @@ -40,7 +63,7 @@ src/ Grep pattern="Gh\|Cargo\|Git\|Grep" path="src/main.rs" output_mode="content" # Step 2: Follow to module -Read file_path="src/gh_cmd.rs" +Read file_path="src/cmds/git/gh_cmd.rs" ``` ### "Where is function X defined?" @@ -52,8 +75,8 @@ Grep pattern="fn filter_git_log\|fn run\b" type="rust" ### "All command modules" ``` -Glob pattern="src/*_cmd.rs" -# Then: src/git.rs, src/runner.rs for non-*_cmd.rs modules +Glob pattern="src/cmds/**/*_cmd.rs" +# Also: src/cmds/git/git.rs, src/cmds/rust/runner.rs, src/cmds/cloud/container.rs ``` ### "Find all lazy_static regex definitions" @@ -92,27 +115,27 @@ Glob pattern="tests/fixtures/*.txt" ### Adding a new filter 1. Check `src/main.rs` for Commands enum structure -2. Check existing `*_cmd.rs` for patterns to follow (e.g., `src/gh_cmd.rs`) -3. Check `src/utils.rs` for shared helpers before reimplementing +2. Check existing modules in `src/cmds//` for patterns to follow (e.g., `src/cmds/git/gh_cmd.rs`) +3. Check `src/core/utils.rs` for shared helpers before reimplementing 4. Check `tests/fixtures/` for existing fixture patterns ### Debugging filter output -1. Start with `src/_cmd.rs` → find `run()` function +1. Start with `src/cmds//_cmd.rs` → find `run()` function 2. Trace filter function (usually `filter_()`) 3. Check `lazy_static!` regex patterns in same file -4. Check `src/utils.rs::strip_ansi()` if ANSI codes involved +4. Check `src/core/utils.rs::strip_ansi()` if ANSI codes involved ### Tracking/metrics issues -1. `src/tracking.rs` → `track_command()` function -2. `src/config.rs` → `tracking.database_path` field +1. `src/core/tracking.rs` → `track_command()` function +2. `src/core/config.rs` → `tracking.database_path` field 3. `RTK_DB_PATH` env var overrides config ### Configuration issues -1. `src/config.rs` → `RtkConfig` struct -2. `src/init.rs` → `rtk init` command +1. `src/core/config.rs` → `RtkConfig` struct +2. `src/hooks/init.rs` → `rtk init` command 3. Config file: `~/.config/rtk/config.toml` 4. Filter files: `~/.config/rtk/filters/` (global) or `.rtk/filters/` (project) @@ -120,7 +143,7 @@ Glob pattern="tests/fixtures/*.txt" ``` Glob pattern=".rtk/filters/*.toml" # Project-local filters -Glob pattern="src/filter_*.rs" # TOML filter engine +Glob pattern="src/core/toml_filter.rs" # TOML filter engine Grep pattern="FilterRule\|FilterConfig" type="rust" ``` diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 7651df48..413e94f0 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -1,10 +1,10 @@ # Copilot Instructions for rtk -**rtk (Rust Token Killer)** is a CLI proxy that filters and compresses command outputs before they reach an LLM context, saving 60–90% of tokens. It wraps common tools (`git`, `cargo`, `grep`, `pnpm`, `go`, etc.) and outputs condensed summaries instead of raw output. +**rtk (Rust Token Killer)** is a CLI proxy that filters and compresses command outputs before they reach an LLM context, saving 60-90% of tokens. It wraps common tools (`git`, `cargo`, `grep`, `pnpm`, `go`, etc.) and outputs condensed summaries instead of raw output. ## Using rtk in this session -**Always prefix commands with `rtk` when running shell commands** — this is the entire point of the project and reduces token consumption for every operation you perform. +**Always prefix commands with `rtk` when running shell commands** — this reduces token consumption for every operation you perform. ```bash # Instead of: Use: @@ -17,7 +17,7 @@ grep -r "pattern" src/ rtk grep -r "pattern" src/ **rtk meta-commands** (always use these directly, no prefix needed): ```bash -rtk gain # Show token savings analytics for this session +rtk gain # Show token savings analytics rtk gain --history # Full command history with per-command savings rtk discover # Scan session history for missed rtk opportunities rtk proxy # Run a command raw (no filtering) but still track it @@ -29,99 +29,38 @@ rtk --version # Should print: rtk X.Y.Z rtk gain # Should show a dashboard (not "command not found") ``` -> ⚠️ **Name collision**: `rtk gain` failing means you have `reachingforthejack/rtk` (Rust Type Kit) installed instead of this project. Run `which rtk` and check the binary source. +> Name collision: `rtk gain` failing means you have `reachingforthejack/rtk` (Rust Type Kit) installed instead. Run `which rtk` to check. ## Build, Test & Lint ```bash -# Development build -cargo build - -# Run all tests -cargo test - -# Run a single test by name -cargo test test_filter_git_log - -# Run all tests in a module -cargo test git::tests:: - -# Run tests with stdout -cargo test -- --nocapture +cargo build # Development build +cargo test # All tests +cargo test test_name # Single test +cargo test module::tests:: # Module tests +cargo test -- --nocapture # With stdout # Pre-commit gate (must all pass before any PR) cargo fmt --all --check && cargo clippy --all-targets && cargo test -# Smoke tests (requires installed binary) -bash scripts/test-all.sh +bash scripts/test-all.sh # Smoke tests (requires installed binary) ``` PRs target the **`develop`** branch, not `main`. All commits require a DCO sign-off (`git commit -s`). ## Architecture -``` -main.rs ← Clap Commands enum → specialized module (git.rs, *_cmd.rs, etc.) - ↓ - execute subprocess - ↓ - filter/compress output - ↓ - tracking::TimedExecution → SQLite (~/.local/share/rtk/tracking.db) -``` - -Key modules: -- **`main.rs`** — Clap `Commands` enum routes every subcommand to its module. Each arm calls `tracking::TimedExecution::start()` before running, then `.track(...)` after. -- **`filter.rs`** — Language-aware filtering with `FilterLevel` (`none` / `minimal` / `aggressive`) and `Language` enum. Used by `read` and `smart` commands. -- **`tracking.rs`** — SQLite persistence for token savings, scoped per project path. Powers `rtk gain`. -- **`tee.rs`** — On filter failure, saves raw output to `~/.local/share/rtk/tee/` and prints a one-line hint so the LLM can re-read without re-running the command. -- **`utils.rs`** — Shared helpers: `truncate`, `strip_ansi`, `execute_command`, package-manager auto-detection (pnpm/yarn/npm/npx). +rtk routes CLI commands via a Clap `Commands` enum in `main.rs` to specialized filter modules in `src/cmds/*/`, each executing the underlying command and compressing output. Token savings are tracked in SQLite via `src/core/tracking.rs`. -New commands follow this structure: one file `src/_cmd.rs` with a `pub fn run(...)` entry point, registered in the `Commands` enum in `main.rs`. +For full details see [ARCHITECTURE.md](../ARCHITECTURE.md) and [docs/TECHNICAL.md](../docs/TECHNICAL.md). Module responsibilities are documented in each folder's `README.md` and each file's `//!` doc header. ## Key Conventions -### Error handling -- Use `anyhow::Result` throughout (this is a binary, not a library). -- Always attach context: `operation.context("description")?` — never bare `?` without context. -- No `unwrap()` in production code; `expect("reason")` is acceptable only in tests. -- Every filter must fall back to raw command execution on error — never break the user's workflow. - -### Regex -- Compile once with `lazy_static!`, never inside a function body: - ```rust - lazy_static! { - static ref RE: Regex = Regex::new(r"pattern").unwrap(); - } - ``` - -### Testing -- Unit tests live **inside the module file** in `#[cfg(test)] mod tests { ... }` — not in `tests/`. -- Fixtures are real captured command output in `tests/fixtures/_raw.txt`, loaded with `include_str!("../tests/fixtures/...")`. -- Each test module defines its own local `fn count_tokens(text: &str) -> usize` (word-split approximation) — there is no shared utility for this. -- Token savings assertions use `assert!(savings >= 60.0, ...)`. -- Snapshot tests use `assert_snapshot!()` from the `insta` crate; review with `cargo insta review`. - -### Adding a new command -1. Create `src/_cmd.rs` with `pub fn run(...)`. -2. Add `mod _cmd;` at the top of `main.rs`. -3. Add a variant to the `Commands` enum with `#[arg(trailing_var_arg = true, allow_hyphen_values = true)]` for pass-through flags. -4. Route the variant in the `match` block, wrapping execution with `tracking::TimedExecution`. -5. Write a fixture from real output, then unit tests in the module file. -6. Update `README.md` (command list + savings %) and `CHANGELOG.md`. - -### Exit codes -Preserve the underlying command's exit code. Use `std::process::exit(code)` when the child process exits non-zero. - -### Performance constraints -- Startup must stay under 10ms — no async runtime (no `tokio`/`async-std`). -- No blocking I/O at startup; config is loaded on-demand. -- Binary size target: <5 MB stripped. - -### Branch naming -``` -fix(scope): short-description -feat(scope): short-description -chore(scope): short-description -``` -`scope` is the affected component (e.g. `git`, `filter`, `tracking`). +- **Error handling**: `anyhow::Result` with `.context("description")?` — no bare `?`, no `unwrap()` in production. Filters must fall back to raw command on error. +- **Regex**: Always `lazy_static!`, never compile inside a function body. +- **Testing**: Unit tests inside modules (`#[cfg(test)] mod tests`). Fixtures in `tests/fixtures/`. Token savings assertions with `count_tokens()`. +- **Exit codes**: Preserve the underlying command's exit code via `std::process::exit(code)`. +- **Performance**: Startup <10ms (no async runtime), binary <5MB stripped. +- **Branch naming**: `fix(scope):`, `feat(scope):`, `chore(scope):` where scope is the affected component. + +For the full contribution workflow, design philosophy, and new-filter checklist, see [CONTRIBUTING.md](../CONTRIBUTING.md). diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 0ae617e1..334a294b 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1,8 +1,8 @@ # rtk Architecture Documentation -> **rtk (Rust Token Killer)** - A high-performance CLI proxy that minimizes LLM token consumption through intelligent output filtering and compression. +> **Deep reference** for RTK's system design, filtering taxonomy, performance characteristics, and architecture decisions. For a guided tour of the end-to-end flow, start with [docs/TECHNICAL.md](docs/TECHNICAL.md). -This document provides a comprehensive architectural overview of rtk, including system design, data flows, module organization, and implementation patterns. +**rtk (Rust Token Killer)** is a high-performance CLI proxy that minimizes LLM token consumption through intelligent output filtering and compression. --- @@ -17,51 +17,16 @@ This document provides a comprehensive architectural overview of rtk, including 7. [Global Flags Architecture](#global-flags-architecture) 8. [Error Handling](#error-handling) 9. [Configuration System](#configuration-system) -10. [Module Development Pattern](#module-development-pattern) +10. [Common Patterns](#common-patterns) 11. [Build Optimizations](#build-optimizations) 12. [Extensibility Guide](#extensibility-guide) +13. [Architecture Decision Records](#architecture-decision-records) --- ## System Overview -### Proxy Pattern Architecture - -``` -┌────────────────────────────────────────────────────────────────────────┐ -│ rtk - Token Optimization Proxy │ -└────────────────────────────────────────────────────────────────────────┘ - -User Input CLI Layer Router Module Layer -────────── ───────── ────── ──────────── - -$ rtk git log ─→ Clap Parser ─→ Commands ─→ git::run() - -v --oneline (main.rs) enum match - • Parse args Execute: git log - • Extract flags Capture output - • Route command ↓ - Filter/Compress - ↓ -$ 3 commits ←─ Terminal ←─ Format ←─ Compact Stats - +142/-89 colored optimized (90% reduction) - output ↓ - tracking::track() - ↓ - SQLite INSERT - (~/.local/share/rtk/) -``` - -### Key Components - -| Component | Location | Responsibility | -|-----------|----------|----------------| -| **CLI Parser** | main.rs | Clap-based argument parsing, global flags | -| **Command Router** | main.rs | Dispatch to specialized modules | -| **Module Layer** | src/*_cmd.rs, src/git.rs, etc. | Command execution + filtering | -| **Shared Utils** | utils.rs | Package manager detection, text processing | -| **Filter Engine** | filter.rs | Language-aware code filtering | -| **Tracking** | tracking.rs | SQLite-based token metrics | -| **Config** | config.rs, init.rs | User preferences, LLM integration | +> For the proxy pattern diagram and key components table, see [docs/TECHNICAL.md](docs/TECHNICAL.md#2-architecture-overview). ### Design Principles @@ -73,39 +38,7 @@ $ 3 commits ←─ Terminal ←─ Format ←─ Compact Sta ### Hook Architecture (v0.9.5+) -The recommended deployment mode uses a Claude Code PreToolUse hook for 100% transparent command rewriting. - -``` -┌────────────────────────────────────────────────────────────────────────┐ -│ Hook-Based Command Rewriting │ -└────────────────────────────────────────────────────────────────────────┘ - -Claude Code settings.json rtk-rewrite.sh RTK binary - │ │ │ │ - │ Bash: "git status" │ │ │ - │ ─────────────────────►│ │ │ - │ │ PreToolUse hook │ │ - │ │ ───────────────────►│ │ - │ │ │ detect: git │ - │ │ │ rewrite: │ - │ │ │ rtk git status │ - │ │◄────────────────────│ │ - │ │ updatedInput │ │ - │ │ │ - │ execute: rtk git status ────────────────────────────────────────► - │ │ run git - │ │ filter - │ │ track - │◄────────────────────────────────────────────────────────────────── - │ "3 modified, 1 untracked ✓" (~10 tokens vs ~200 raw) - │ - │ Claude never sees the rewrite — it only sees optimized output. - -Files: - ~/.claude/hooks/rtk-rewrite.sh ← thin delegator (calls `rtk rewrite`, ~50 lines) - ~/.claude/settings.json ← hook registry (PreToolUse registration) - ~/.claude/RTK.md ← minimal context hint (10 lines) -``` +> For the hook interception diagram and agent-specific JSON formats, see [docs/TECHNICAL.md](docs/TECHNICAL.md#32-hook-interception-command-rewriting) and [hooks/README.md](hooks/README.md). Two hook strategies: @@ -224,85 +157,33 @@ Database: ~/.local/share/rtk/history.db ## Module Organization -### Complete Module Map (30 Modules) - -``` -┌────────────────────────────────────────────────────────────────────────┐ -│ Module Organization │ -└────────────────────────────────────────────────────────────────────────┘ - -Category Module Commands Savings File -────────────────────────────────────────────────────────────────────────── - -GIT git.rs status, diff, log 85-99% ✓ - add, commit, push - branch, checkout +### Module Map -CODE SEARCH grep_cmd.rs grep 60-80% ✓ - diff_cmd.rs diff 70-85% ✓ - find_cmd.rs find 50-70% ✓ +> For the full file-level module tree, see [docs/TECHNICAL.md](docs/TECHNICAL.md#4-folder-map) and each folder's README. -FILE OPS ls.rs ls 50-70% ✓ - read.rs read 40-90% ✓ +**Token savings by ecosystem:** -EXECUTION runner.rs err, test 60-99% ✓ - summary.rs smart (heuristic) 50-80% ✓ - local_llm.rs smart (LLM mode) 60-90% ✓ - -LOGS/DATA log_cmd.rs log 70-90% ✓ - json_cmd.rs json 80-95% ✓ - -JS/TS STACK lint_cmd.rs lint 84% ✓ - tsc_cmd.rs tsc 83% ✓ - next_cmd.rs next 87% ✓ - prettier_cmd.rs prettier 70% ✓ - playwright_cmd.rs playwright 94% ✓ - prisma_cmd.rs prisma 88% ✓ - vitest_cmd.rs vitest 99.5% ✓ - pnpm_cmd.rs pnpm 70-90% ✓ - -CONTAINERS container.rs podman, docker 60-80% ✓ - -VCS gh_cmd.rs gh 26-87% ✓ - -PYTHON ruff_cmd.rs ruff check/format 80%+ ✓ - pytest_cmd.rs pytest 90%+ ✓ - pip_cmd.rs pip list/outdated 70-85% ✓ - -GO go_cmd.rs go test/build/vet 75-90% ✓ - golangci_cmd.rs golangci-lint 85% ✓ - -RUBY rake_cmd.rs rake/rails test 85-90% ✓ - rspec_cmd.rs rspec 60%+ ✓ - rubocop_cmd.rs rubocop 60%+ ✓ - -NETWORK wget_cmd.rs wget 85-95% ✓ - curl_cmd.rs curl 70% ✓ - -INFRA aws_cmd.rs aws 80% ✓ - psql_cmd.rs psql 75% ✓ - -DEPENDENCIES deps.rs deps 80-90% ✓ - -ENVIRONMENT env_cmd.rs env 60-80% ✓ - -SYSTEM init.rs init N/A ✓ - gain.rs gain N/A ✓ - config.rs (internal) N/A ✓ - rewrite_cmd.rs rewrite N/A ✓ - -SHARED utils.rs Helpers N/A ✓ - filter.rs Language filters N/A ✓ - tracking.rs Token tracking N/A ✓ - tee.rs Full output recovery N/A ✓ +``` +Savings by ecosystem: + GIT (cmds/git/) 85-99% status, diff, log, gh, gt + JS/TS (cmds/js/) 70-99% lint, tsc, next, prettier, playwright, prisma, vitest, pnpm + PYTHON (cmds/python/) 70-90% ruff, pytest, mypy, pip + GO (cmds/go/) 75-90% go test/build/vet, golangci-lint + RUBY (cmds/ruby/) 60-90% rake, rspec, rubocop + DOTNET (cmds/dotnet/) 70-85% dotnet build/test, binlog + CLOUD (cmds/cloud/) 60-80% aws, docker/kubectl, curl, wget, psql + SYSTEM (cmds/system/) 50-90% ls, tree, read, grep, find, json, log, env, deps + RUST (cmds/rust/) 60-99% cargo test/build/clippy, err ``` **Total: 67 modules** (45 command modules + 22 infrastructure modules) ### Module Count Breakdown -- **Command Modules**: 45 (directly exposed to users) -- **Infrastructure Modules**: 22 (utils, filter, tracking, tee, config, init, gain, toml_filter, verify_cmd, trust, etc.) +- **Command Modules**: 45 in `src/cmds/` (directly exposed to users) +- **Core Infrastructure**: 8 in `src/core/` (utils, filter, tracking, tee, config, toml_filter, display_helpers, telemetry) +- **Hook System**: 8 in `src/hooks/` (init, rewrite, hook_cmd, hook_check, hook_audit, verify, trust, integrity) +- **Analytics**: 4 in `src/analytics/` (gain, cc_economics, ccusage, session_cmd) - **Git Commands**: 7 operations (status, diff, log, add, commit, push, branch/checkout) - **JS/TS Tooling**: 8 modules (modern frontend/fullstack development) - **Python Tooling**: 3 modules (ruff, pytest, pip) @@ -422,7 +303,7 @@ Strategy Modules Technique Reduction Used by: go test (NDJSON stream, interleaved package events) ``` -### Code Filtering Levels (filter.rs) +### Code Filtering Levels (src/core/filter.rs) ```rust // FilterLevel::None - Keep everything @@ -576,29 +457,7 @@ golangci_cmd.rs JSON PARSING JSON API 85% #### Sub-Enum Pattern (go_cmd.rs) -```rust -// main.rs enum definition -Commands::Go { - #[command(subcommand)] - command: GoCommand, -} - -// go_cmd.rs sub-enum -pub enum GoCommand { - Test { args: Vec }, - Build { args: Vec }, - Vet { args: Vec }, -} - -// Router -pub fn run(command: &GoCommand, verbose: u8) -> Result<()> { - match command { - GoCommand::Test { args } => run_test(args, verbose), - GoCommand::Build { args } => run_build(args, verbose), - GoCommand::Vet { args } => run_vet(args, verbose), - } -} -``` +Uses `Commands::Go { #[command(subcommand)] command: GoCommand }` in main.rs, with `GoCommand` enum routing to `run_test/run_build/run_vet`. Mirrors git/cargo patterns. **Why Sub-Enum?** - `go test/build/vet` are semantically related (core Go toolchain) @@ -664,38 +523,6 @@ Output format known? Examples: go vet, go build ``` -### Testing Patterns - -#### Python Module Tests - -```rust -// pytest_cmd.rs tests -#[test] -fn test_pytest_state_machine() { - let output = "test_auth.py::test_login PASSED\ntest_db.py::test_query FAILED"; - let result = parse_pytest_output(output); - assert!(result.contains("1 failed")); - assert!(result.contains("test_db.py::test_query")); -} -``` - -#### Go Module Tests - -```rust -// go_cmd.rs tests -#[test] -fn test_go_test_ndjson_interleaved() { - let output = r#"{"Action":"run","Package":"pkg1"} -{"Action":"fail","Package":"pkg1","Test":"TestA"} -{"Action":"run","Package":"pkg2"} -{"Action":"pass","Package":"pkg2","Test":"TestB"}"#; - - let result = parse_go_test_ndjson(output); - assert!(result.contains("pkg1: 1 failed")); - assert!(!result.contains("pkg2")); // pkg2 passed, hidden -} -``` - ### Performance Characteristics ``` @@ -738,26 +565,9 @@ When adding Python/Go module support: ## Shared Infrastructure -### Utilities Layer (utils.rs) - -``` -┌────────────────────────────────────────────────────────────────────────┐ -│ Shared Utilities Layer │ -└────────────────────────────────────────────────────────────────────────┘ - -utils.rs provides common functionality: - -┌─────────────────────────────────────────┐ -│ truncate(s: &str, max: usize) → String │ Text truncation with "..." -├─────────────────────────────────────────┤ -│ strip_ansi(text: &str) → String │ Remove ANSI color codes -├─────────────────────────────────────────┤ -│ execute_command(cmd, args) │ Shell execution helper -│ → (stdout, stderr, exit_code) │ with error context -└─────────────────────────────────────────┘ +### Utilities Layer -Used by: All command modules (24 modules depend on utils.rs) -``` +> For the full utilities API (`truncate`, `strip_ansi`, `execute_command`, `ruby_exec`, etc.), see [src/core/README.md](src/core/README.md). Used by 41 command modules. ### Package Manager Detection Pattern @@ -918,15 +728,7 @@ Flow: ### Thread Safety -```rust -// tracking.rs:9-11 -lazy_static::lazy_static! { - static ref TRACKER: Mutex> = Mutex::new(None); -} -``` - -**Design**: Single-threaded execution with Mutex for future-proofing. -**Current State**: No multi-threading, but Mutex enables safe concurrent access if needed. +Single-threaded execution with `Mutex>` for future-proofing. No multi-threading currently, but safe concurrent access is possible if needed. --- @@ -1064,43 +866,11 @@ Modules with Exit Code Preservation: ## Configuration System -### Two-Tier Configuration - -``` -┌────────────────────────────────────────────────────────────────────────┐ -│ Configuration Architecture │ -└────────────────────────────────────────────────────────────────────────┘ - -1. User Settings (config.toml) - ─────────────────────────── - Location: ~/.config/rtk/config.toml - - Format: - [general] - default_filter_level = "minimal" - enable_tracking = true - retention_days = 90 - - Loaded by: config.rs (main.rs:650-656) - -2. LLM Integration (CLAUDE.md) - ──────────────────────────── - Locations: - • Global: ~/.config/rtk/CLAUDE.md - • Local: ./CLAUDE.md (project-specific) +### Configuration - Purpose: Instruct LLM (Claude Code) to use rtk prefix - Created by: rtk init [--global] +> For config file format, tee settings, tracking database path, and TOML filter tiers, see [src/core/README.md](src/core/README.md). - Template (init.rs:40-60): - # CLAUDE.md - Use `rtk` prefix for all commands: - - rtk git status - - rtk grep "pattern" - - rtk read file.rs - - Benefits: 60-90% token reduction -``` +Two tiers: **User settings** (`~/.config/rtk/config.toml`) and **LLM integration** (CLAUDE.md via `rtk init`). ### Initialization Flow @@ -1137,85 +907,7 @@ Success: "✓ Initialized rtk for LLM integration" --- -## Module Development Pattern - -### Standard Module Template - -```rust -// src/example_cmd.rs - -use anyhow::{Context, Result}; -use std::process::Command; -use crate::{tracking, utils}; - -/// Public entry point called by main.rs router -pub fn run(args: &[String], verbose: u8) -> Result<()> { - // 1. Execute underlying command - let raw_output = execute_command(args)?; - - // 2. Apply filtering strategy - let filtered = filter_output(&raw_output, verbose); - - // 3. Print result - println!("{}", filtered); - - // 4. Track token savings - tracking::track( - "original_command", - "rtk command", - &raw_output, - &filtered - ); - - Ok(()) -} - -/// Execute the underlying tool -fn execute_command(args: &[String]) -> Result { - let output = Command::new("tool") - .args(args) - .output() - .context("Failed to execute tool")?; - - // Preserve exit codes (critical for CI/CD) - if !output.status.success() { - let stderr = String::from_utf8_lossy(&output.stderr); - eprintln!("{}", stderr); - std::process::exit(output.status.code().unwrap_or(1)); - } - - Ok(String::from_utf8_lossy(&output.stdout).to_string()) -} - -/// Apply filtering strategy -fn filter_output(raw: &str, verbose: u8) -> String { - // Choose strategy: stats, grouping, deduplication, etc. - // See "Filtering Strategies" section for options - - if verbose >= 3 { - eprintln!("Raw output:\n{}", raw); - } - - // Apply compression logic - let compressed = compress(raw); - - compressed -} - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn test_filter_output() { - let raw = "verbose output here"; - let filtered = filter_output(raw, 0); - assert!(filtered.len() < raw.len()); - } -} -``` - -### Common Patterns +## Common Patterns #### 1. Package Manager Detection (JS/TS modules) @@ -1234,18 +926,7 @@ let mut cmd = if is_pnpm { }; ``` -#### 2. Lazy Static Regex (filter.rs, runner.rs) - -```rust -lazy_static::lazy_static! { - static ref PATTERN: Regex = Regex::new(r"ERROR:.*").unwrap(); -} - -// Usage: compiled once, reused across invocations -let matches: Vec<_> = PATTERN.find_iter(text).collect(); -``` - -#### 3. Verbosity Guards +#### 2. Verbosity Guards ```rust if verbose > 0 { @@ -1308,162 +989,11 @@ Overhead Sources: • SQLite tracking: ~1-3ms ``` -### Compilation - -```bash -# Development build (fast compilation, debug symbols) -cargo build - -# Release build (optimized, stripped) -cargo build --release - -# Check without building (fast feedback) -cargo check - -# Run tests -cargo test - -# Lint with clippy -cargo clippy --all-targets - -# Format code -cargo fmt -``` - --- ## Extensibility Guide -### Adding a New Command - -**Step-by-step process to add a new rtk command:** - -#### 1. Create Module File - -```bash -touch src/mycmd.rs -``` - -#### 2. Implement Module (src/mycmd.rs) - -```rust -use anyhow::{Context, Result}; -use std::process::Command; -use crate::tracking; - -pub fn run(args: &[String], verbose: u8) -> Result<()> { - // Execute underlying command - let output = Command::new("mycmd") - .args(args) - .output() - .context("Failed to execute mycmd")?; - - let raw = String::from_utf8_lossy(&output.stdout); - - // Apply filtering strategy - let filtered = filter(&raw, verbose); - - // Print result - println!("{}", filtered); - - // Track savings - tracking::track("mycmd", "rtk mycmd", &raw, &filtered); - - Ok(()) -} - -fn filter(raw: &str, verbose: u8) -> String { - // Implement your filtering logic - raw.lines().take(10).collect::>().join("\n") -} - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn test_filter() { - let raw = "line1\nline2\n"; - let result = filter(raw, 0); - assert!(result.contains("line1")); - } -} -``` - -#### 3. Declare Module (main.rs) - -```rust -// Add to module declarations (alphabetically) -mod mycmd; -``` - -#### 4. Add Command Enum Variant (main.rs) - -```rust -#[derive(Subcommand)] -enum Commands { - // ... existing commands ... - - /// Description of your command - Mycmd { - /// Arguments your command accepts - #[arg(trailing_var_arg = true, allow_hyphen_values = true)] - args: Vec, - }, -} -``` - -#### 5. Add Router Match Arm (main.rs) - -```rust -match cli.command { - // ... existing matches ... - - Commands::Mycmd { args } => { - mycmd::run(&args, verbose)?; - } -} -``` - -#### 6. Test Your Command - -```bash -# Build and test -cargo build -./target/debug/rtk mycmd arg1 arg2 - -# Run tests -cargo test mycmd::tests - -# Check with clippy -cargo clippy --all-targets -``` - -#### 7. Document Your Command - -Update CLAUDE.md: - -```markdown -### New Commands - -**rtk mycmd** - Description of what it does -- Strategy: [stats/grouping/filtering/etc.] -- Savings: X-Y% -- Used by: [workflow description] -``` - -### Design Checklist - -When implementing a new command, consider: - -- [ ] **Filtering Strategy**: Which of the 9 strategies fits best? -- [ ] **Exit Code Preservation**: Does your command need to preserve exit codes for CI/CD? -- [ ] **Verbosity Support**: Add debug output for `-v`, `-vv`, `-vvv` -- [ ] **Error Handling**: Use `.context()` for meaningful error messages -- [ ] **Package Manager Detection**: For JS/TS tools, use the standard detection pattern -- [ ] **Tests**: Add unit tests for filtering logic -- [ ] **Token Tracking**: Integrate with `tracking::track()` -- [ ] **Documentation**: Update CLAUDE.md with token savings and use cases +> For the complete step-by-step process to add a new command (module file, enum variant, routing, tests, documentation), see [CONTRIBUTING.md](CONTRIBUTING.md#complete-contribution-checklist) and the [Standard Module Template in cmds/README.md](src/cmds/README.md). --- @@ -1500,11 +1030,11 @@ When implementing a new command, consider: ## Resources +- **[docs/TECHNICAL.md](docs/TECHNICAL.md)**: Guided tour of end-to-end flow +- **[CONTRIBUTING.md](CONTRIBUTING.md)**: Design philosophy, contribution workflow, checklist +- **CLAUDE.md**: Quick reference for AI agents (dev commands, build verification) - **README.md**: User guide, installation, examples -- **CLAUDE.md**: Developer documentation, module details, PR history - **Cargo.toml**: Dependencies, build profiles, package metadata -- **src/**: Source code organized by module -- **.github/workflows/**: CI/CD automation (multi-platform builds, releases) --- @@ -1522,6 +1052,5 @@ When implementing a new command, consider: --- -**Last Updated**: 2026-02-22 -**Architecture Version**: 2.2 -**rtk Version**: 0.28.2 +**Last Updated**: 2026-03-24 +**Architecture Version**: 3.1 diff --git a/CLAUDE.md b/CLAUDE.md index 35ff19ed..2ecd1d3c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -8,11 +8,11 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co This is a fork with critical fixes for git argument parsing and modern JavaScript stack support (pnpm, vitest, Next.js, TypeScript, Playwright, Prisma). -### ⚠️ Name Collision Warning +### Name Collision Warning **Two different "rtk" projects exist:** -- ✅ **This project**: Rust Token Killer (rtk-ai/rtk) -- ❌ **reachingforthejack/rtk**: Rust Type Kit (DIFFERENT - generates Rust types) +- This project: Rust Token Killer (rtk-ai/rtk) +- reachingforthejack/rtk: Rust Type Kit (DIFFERENT - generates Rust types) **Verify correct installation:** ```bash @@ -29,133 +29,51 @@ If `rtk gain` fails, you have the wrong package installed. ### Build & Run ```bash -# Development build cargo build # raw rtk cargo build # preferred (token-optimized) - -# Release build (optimized) -cargo build --release -rtk cargo build --release - -# Run directly -cargo run -- - -# Install locally -cargo install --path . +cargo build --release # release build (optimized) +cargo run -- # run directly +cargo install --path . # install locally ``` ### Testing ```bash -# Run all tests -cargo test # raw +cargo test # all tests rtk cargo test # preferred (token-optimized) - -# Run specific test -cargo test -rtk cargo test - -# Run tests with output -cargo test -- --nocapture -rtk cargo test -- --nocapture - -# Run tests in specific module -cargo test :: -rtk cargo test :: +cargo test # specific test +cargo test :: # module tests +cargo test -- --nocapture # with stdout +bash scripts/test-all.sh # smoke tests (installed binary required) ``` ### Linting & Quality ```bash -# Check without building -cargo check # raw -rtk cargo check # preferred (token-optimized) - -# Format code -cargo fmt # passthrough (0% savings, but works) - -# Run clippy lints -cargo clippy # raw -rtk cargo clippy # preferred (token-optimized) +cargo check # check without building +cargo fmt # format code +cargo clippy --all-targets # all clippy lints +rtk cargo clippy --all-targets # preferred +``` -# Check all targets -cargo clippy --all-targets -rtk cargo clippy --all-targets +### Pre-commit Gate +```bash +cargo fmt --all && cargo clippy --all-targets && cargo test --all ``` ### Package Building ```bash -# Build DEB package (Linux) -cargo install cargo-deb -cargo deb - -# Build RPM package (Fedora/RHEL) -cargo install cargo-generate-rpm -cargo build --release -cargo generate-rpm +cargo deb # DEB package (needs cargo-deb) +cargo generate-rpm # RPM package (needs cargo-generate-rpm, after release build) ``` ## Architecture -### Core Design Pattern - -rtk uses a **command proxy architecture** with specialized modules for each output type: +rtk uses a **command proxy architecture**: `main.rs` routes CLI commands via a Clap `Commands` enum to specialized filter modules in `src/cmds/*/`, each of which executes the underlying command and compresses its output. Token savings are tracked in SQLite via `src/core/tracking.rs`. -``` -main.rs (CLI entry) - → Clap command parsing - → Route to specialized modules - → tracking.rs (SQLite) records token savings -``` +For the full architecture, component details, and module development patterns, see: +- [ARCHITECTURE.md](ARCHITECTURE.md) — System design, module organization, filtering strategies, error handling +- [docs/TECHNICAL.md](docs/TECHNICAL.md) — End-to-end flow, folder map, hook system, filter pipeline -### Key Architectural Components - -**1. Command Modules** (src/*_cmd.rs, src/git.rs, src/container.rs) -- Each module handles a specific command type (git, grep, etc.) -- Responsible for executing underlying commands and transforming output -- Implement token-optimized formatting strategies - -**2. Core Filtering** (src/filter.rs) -- Language-aware code filtering (Rust, Python, JavaScript, etc.) -- Filter levels: `none`, `minimal`, `aggressive` -- Strips comments, whitespace, and function bodies (aggressive mode) -- Used by `read` and `smart` commands - -**3. Token Tracking** (src/tracking.rs) -- SQLite-based persistent storage (~/.local/share/rtk/tracking.db) -- Records: original_cmd, rtk_cmd, input_tokens, output_tokens, savings_pct -- 90-day retention policy with automatic cleanup -- Powers the `rtk gain` analytics command -- **Configurable database path**: Via `RTK_DB_PATH` env var or `config.toml` - - Priority: env var > config file > default location - -**4. Configuration System** (src/config.rs, src/init.rs) -- Manages CLAUDE.md initialization (global vs local) -- Reads ~/.config/rtk/config.toml for user preferences -- `rtk init` command bootstraps LLM integration -- **New**: `tracking.database_path` field for custom DB location - -**5. Tee Output Recovery** (src/tee.rs) -- Saves raw unfiltered output to `~/.local/share/rtk/tee/` on command failure -- Prints one-line hint `[full output: ~/.local/share/rtk/tee/...]` so LLMs can read instead of re-run -- Configurable via `[tee]` section in config.toml or env vars (`RTK_TEE`, `RTK_TEE_DIR`) -- Default mode: failures only, skip outputs < 500 chars, 20 file rotation, 1MB cap -- Silent error handling: tee failure never affects command output or exit code - -**6. Shared Utilities** (src/utils.rs) -- Common functions for command modules: truncate, strip_ansi, execute_command -- Package manager auto-detection (pnpm/yarn/npm/npx) -- Consistent error handling and output formatting -- Used by all modern JavaScript/TypeScript tooling commands - -### Command Routing Flow - -All commands follow this pattern: -```rust -main.rs:Commands enum - → match statement routes to module - → module::run() executes logic - → tracking::track_command() records metrics - → Result<()> propagates errors -``` +Module responsibilities are documented in each folder's `README.md` and each file's `//!` doc header. Browse `src/cmds/*/` to discover available filters. ### Proxy Mode @@ -167,293 +85,30 @@ main.rs:Commands enum - **Bypass RTK filtering**: Workaround bugs or get full unfiltered output - **Track usage metrics**: Measure which commands Claude uses most (visible in `rtk gain --history`) - **Guaranteed compatibility**: Always works even if RTK doesn't implement the command -- **Prototyping**: Test new commands before implementing optimized filtering **Examples**: ```bash -# Full git log output (no truncation) -rtk proxy git log --oneline -20 - -# Raw npm output (no filtering) -rtk proxy npm install express - -# Any command works -rtk proxy curl https://api.example.com/data - -# Tracking shows 0% savings (expected) -rtk gain --history | grep proxy -``` - -**Tracking**: All proxy commands appear in `rtk gain --history` with 0% savings (input = output) but preserve usage statistics. - -### Critical Implementation Details - -**Git Argument Handling** (src/git.rs) -- Uses `trailing_var_arg = true` + `allow_hyphen_values = true` to properly handle git flags -- Auto-detects `--merges` flag to avoid conflicting with `--no-merges` injection -- Propagates git exit codes for CI/CD reliability (PR #5 fix) - -**Output Filtering Strategy** -- Compact mode: Show only summary/failures -- Full mode: Available with `-v` verbosity flags -- Test output: Show only failures (90% token reduction) -- Git operations: Ultra-compressed confirmations ("ok ✓") - -**Language Detection** (src/filter.rs) -- File extension-based with fallback heuristics -- Supports Rust, Python, JS/TS, Java, Go, C/C++, etc. -- Tokenization rules vary by language (comments, strings, blocks) - -### Module Responsibilities - -| Module | Purpose | Token Strategy | -|--------|---------|----------------| -| git.rs | Git operations | Stat summaries + compact diffs | -| grep_cmd.rs | Code search | Group by file, truncate lines | -| ls.rs | Directory listing | Tree format, aggregate counts | -| read.rs | File reading | Filter-level based stripping | -| runner.rs | Command execution | Stderr only (err), failures only (test) | -| log_cmd.rs | Log parsing | Deduplication with counts | -| json_cmd.rs | JSON inspection | Structure without values | -| lint_cmd.rs | ESLint/Biome linting | Group by rule, file summary (84% reduction) | -| tsc_cmd.rs | TypeScript compiler | Group by file/error code (83% reduction) | -| next_cmd.rs | Next.js build/dev | Route metrics, bundle stats only (87% reduction) | -| prettier_cmd.rs | Format checking | Files needing changes only (70% reduction) | -| playwright_cmd.rs | E2E test results | Failures only, grouped by suite (94% reduction) | -| prisma_cmd.rs | Prisma CLI | Strip ASCII art and verbose output (88% reduction) | -| gh_cmd.rs | GitHub CLI | Compact PR/issue/run views (26-87% reduction) | -| vitest_cmd.rs | Vitest test runner | Failures only with ANSI stripping (99.5% reduction) | -| pnpm_cmd.rs | pnpm package manager | Compact dependency trees (70-90% reduction) | -| ruff_cmd.rs | Ruff linter/formatter | JSON for check, text for format (80%+ reduction) | -| pytest_cmd.rs | Pytest test runner | State machine text parser (90%+ reduction) | -| mypy_cmd.rs | Mypy type checker | Group by file/error code (80% reduction) | -| pip_cmd.rs | pip/uv package manager | JSON parsing, auto-detect uv (70-85% reduction) | -| go_cmd.rs | Go commands | NDJSON for test, text for build/vet (80-90% reduction) | -| golangci_cmd.rs | golangci-lint | JSON parsing, group by rule (85% reduction) | -| rake_cmd.rs | Minitest via rake/rails test | State machine text parser, failures only (85-90% reduction) | -| rspec_cmd.rs | RSpec test runner | JSON injection + text fallback, failures only (60%+ reduction) | -| rubocop_cmd.rs | RuboCop linter | JSON injection, group by cop/severity (60%+ reduction) | -| tee.rs | Full output recovery | Save raw output to file on failure, print hint for LLM re-read | -| utils.rs | Shared utilities | Package manager detection, ruby_exec, common formatting | -| discover/ | Claude Code history analysis | Scan JSONL sessions, classify commands, report missed savings | - -## Performance Constraints - -RTK has **strict performance targets** to maintain zero-overhead CLI experience: - -| Metric | Target | Verification Method | -|--------|--------|---------------------| -| **Startup time** | <10ms | `hyperfine 'rtk git status' 'git status'` | -| **Memory overhead** | <5MB resident | `/usr/bin/time -l rtk git status` (macOS) | -| **Token savings** | 60-90% | Verify in tests with `count_tokens()` assertions | -| **Binary size** | <5MB stripped | `ls -lh target/release/rtk` | - -**Performance regressions are release blockers** - always benchmark before/after changes: - -```bash -# Before changes -hyperfine 'rtk git log -10' --warmup 3 > /tmp/before.txt - -# After changes -cargo build --release -hyperfine 'target/release/rtk git log -10' --warmup 3 > /tmp/after.txt - -# Compare (should be <10ms) -diff /tmp/before.txt /tmp/after.txt -``` - -**Why <10ms matters**: Claude Code users expect CLI tools to be instant. Any perceptible delay (>10ms) breaks the developer flow. RTK achieves this through: -- **Zero async overhead**: Single-threaded, no tokio runtime -- **Lazy regex compilation**: Compile once with `lazy_static!`, reuse forever -- **Minimal allocations**: Borrow over clone, in-place filtering -- **No user config**: Zero file I/O on startup (config loaded on-demand) - -## Error Handling - -RTK follows Rust best practices for error handling: - -**Rules**: -- **anyhow::Result** for CLI binary (RTK is an application, not a library) -- **ALWAYS** use `.context("description")` with `?` operator -- **NO unwrap()** in production code (tests only - use `expect("explanation")` if needed) -- **Graceful degradation**: If filter fails, fallback to raw command execution - -**Example**: - -```rust -use anyhow::{Context, Result}; - -pub fn filter_git_log(input: &str) -> Result { - let lines: Vec<_> = input - .lines() - .filter(|line| !line.is_empty()) - .collect(); - - // ✅ RIGHT: Context on error - let hash = extract_hash(lines[0]) - .context("Failed to extract commit hash from git log")?; - - // ❌ WRONG: No context - let hash = extract_hash(lines[0])?; - - // ❌ WRONG: Panic in production - let hash = extract_hash(lines[0]).unwrap(); - - Ok(format!("Commit: {}", hash)) -} -``` - -**Fallback pattern** (critical for all filters): - -```rust -// ✅ RIGHT: Fallback to raw command if filter fails -pub fn execute_with_filter(cmd: &str, args: &[&str]) -> Result<()> { - match get_filter(cmd) { - Some(filter) => match filter.apply(cmd, args) { - Ok(output) => println!("{}", output), - Err(e) => { - eprintln!("Filter failed: {}, falling back to raw", e); - execute_raw(cmd, args)?; - } - }, - None => execute_raw(cmd, args)?, - } - Ok(()) -} - -// ❌ WRONG: Panic if no filter -pub fn execute_with_filter(cmd: &str, args: &[&str]) -> Result<()> { - let filter = get_filter(cmd).expect("Filter must exist"); - filter.apply(cmd, args)?; - Ok(()) -} -``` - -## Common Pitfalls - -**Don't add async dependencies** (kills startup time) -- RTK is single-threaded by design -- Adding tokio/async-std adds ~5-10ms startup overhead -- Use blocking I/O with fallback to raw command - -**Don't recompile regex at runtime** (kills performance) -- ❌ WRONG: `let re = Regex::new(r"pattern").unwrap();` inside function -- ✅ RIGHT: `lazy_static! { static ref RE: Regex = Regex::new(r"pattern").unwrap(); }` - -**Don't panic on filter failure** (breaks user workflow) -- Always fallback to raw command execution -- Log error to stderr, execute original command unchanged - -**Don't assume command output format** (breaks across versions) -- Test with real fixtures from multiple versions -- Use flexible regex patterns that tolerate format changes - -**Don't skip cross-platform testing** (macOS ≠ Linux ≠ Windows) -- Shell escaping differs: bash/zsh vs PowerShell -- Path separators differ: `/` vs `\` -- Line endings differ: LF vs CRLF - -**Don't break pipe compatibility** (users expect Unix behavior) -- `rtk git status | grep modified` must work -- Preserve stdout/stderr separation -- Respect exit codes (0 = success, non-zero = failure) - -## Fork-Specific Features - -### PR #5: Git Argument Parsing Fix (CRITICAL) -- **Problem**: Git flags like `--oneline`, `--cached` were rejected -- **Solution**: Fixed Clap parsing with proper trailing_var_arg configuration -- **Impact**: All git commands now accept native git flags - -### PR #6: pnpm Support -- **New Commands**: `rtk pnpm list`, `rtk pnpm outdated`, `rtk pnpm install` -- **Token Savings**: 70-90% reduction on package manager operations -- **Security**: Package name validation prevents command injection - -### PR #9: Modern JavaScript/TypeScript Tooling (2026-01-29) -- **New Commands**: 6 commands for T3 Stack workflows - - `rtk lint`: ESLint/Biome with grouped rule violations (84% reduction) - - `rtk tsc`: TypeScript compiler errors grouped by file/code (83% reduction) - - `rtk next`: Next.js build with route/bundle metrics (87% reduction) - - `rtk prettier`: Format checker showing files needing changes (70% reduction) - - `rtk playwright`: E2E test results showing failures only (94% reduction) - - `rtk prisma`: Prisma CLI without ASCII art (88% reduction) -- **Shared Infrastructure**: utils.rs module for package manager auto-detection -- **Features**: Exit code preservation, error grouping, consistent formatting -- **Testing**: Validated on a production T3 Stack project - -### Python & Go Support (2026-02-12) -- **Python Commands**: 3 commands for Python development workflows - - `rtk ruff check/format`: Ruff linter/formatter with JSON (check) and text (format) parsing (80%+ reduction) - - `rtk pytest`: Pytest test runner with state machine text parser (90%+ reduction) - - `rtk pip list/outdated/install`: pip package manager with auto-detect uv (70-85% reduction) -- **Go Commands**: 4 commands via sub-enum for Go ecosystem - - `rtk go test`: NDJSON line-by-line parser for interleaved events (90%+ reduction) - - `rtk go build`: Text filter showing errors only (80% reduction) - - `rtk go vet`: Text filter for issues (75% reduction) - - `rtk golangci-lint`: JSON parsing grouped by rule (85% reduction) -- **Architecture**: Standalone Python commands (mirror lint/prettier), Go sub-enum (mirror git/cargo) -- **Patterns**: JSON for structured output (ruff check, golangci-lint, pip), NDJSON streaming (go test), text state machine (pytest), text filters (go build/vet, ruff format) - -### Ruby on Rails Support (2026-03-15) -- **Ruby Commands**: 3 modules for Ruby/Rails development - - `rtk rspec`: RSpec test runner with JSON injection (`--format json`), text fallback (60%+ reduction) - - `rtk rubocop`: RuboCop linter with JSON injection, group by cop/severity (60%+ reduction) - - `rtk rake test`: Minitest filter via rake/rails test, state machine parser (85-90% reduction) -- **TOML Filter**: `bundle-install.toml` for bundle install/update — strips `Using` lines (90%+ reduction) -- **Shared Infrastructure**: `ruby_exec()` in utils.rs auto-detects `bundle exec` when Gemfile exists -- **Hook Integration**: Rewrites `rspec`, `rubocop`, `rake test`, `rails test`, `bundle exec` variants - -## Testing Strategy - -### TDD Workflow (mandatory) -All code follows Red-Green-Refactor. See `.claude/skills/rtk-tdd/` for the full workflow and Rust-idiomatic patterns. See `.claude/skills/rtk-tdd/references/testing-patterns.md` for RTK-specific patterns and untested module backlog. - -### Test Architecture -- **Unit tests**: Embedded `#[cfg(test)] mod tests` in each module (105+ tests, 25+ files) -- **Smoke tests**: `scripts/test-all.sh` (69 assertions on all commands) -- **Dominant pattern**: raw string input -> filter function -> assert output contains/excludes - -### Pre-commit gate -```bash -cargo fmt --all --check && rtk cargo clippy --all-targets && rtk cargo test +rtk proxy git log --oneline -20 # Full git log output (no truncation) +rtk proxy npm install express # Raw npm output (no filtering) +rtk proxy curl https://api.example.com/data # Any command works ``` -### Test commands -```bash -cargo test # All tests -cargo test filter::tests:: # Module-specific -cargo test -- --nocapture # With stdout -bash scripts/test-all.sh # Smoke tests (installed binary required) -``` +All proxy commands appear in `rtk gain --history` with 0% savings (input = output). -## Dependencies +## Coding Rules -Core dependencies (see Cargo.toml): -- **clap**: CLI parsing with derive macros -- **anyhow**: Error handling -- **rusqlite**: SQLite for tracking database -- **regex**: Pattern matching for filtering -- **ignore**: gitignore-aware file traversal -- **colored**: Terminal output formatting -- **serde/serde_json**: Configuration and JSON parsing +Rust patterns, error handling, and anti-patterns are defined in `.claude/rules/rust-patterns.md` (auto-loaded into context). Key points: -## Build Optimizations +- **anyhow::Result** everywhere, always `.context("description")?` +- **No unwrap()** in production code +- **lazy_static!** for all regex (never compile inside a function) +- **Fallback pattern**: if filter fails, execute raw command unchanged +- **No async**: single-threaded by design (startup <10ms) +- **Exit code propagation**: `std::process::exit(code)` on child failure -Release profile (Cargo.toml:31-36): -- `opt-level = 3`: Maximum optimization -- `lto = true`: Link-time optimization -- `codegen-units = 1`: Single codegen for better optimization -- `strip = true`: Remove debug symbols -- `panic = "abort"`: Smaller binary size +Testing strategy and performance targets are defined in `.claude/rules/cli-testing.md` (auto-loaded). Key targets: <10ms startup, <5MB memory, 60-90% token savings. -## CI/CD - -GitHub Actions workflow (.github/workflows/release.yml): -- Multi-platform builds (macOS, Linux x86_64/ARM64, Windows) -- DEB/RPM package generation -- Automated releases on version tags (v*) -- Checksums for binary verification +For contribution workflow, design philosophy, and the complete checklist for adding new filters, see [CONTRIBUTING.md](CONTRIBUTING.md). ## Build Verification (Mandatory) @@ -467,55 +122,14 @@ cargo fmt --all && cargo clippy --all-targets && cargo test --all - Never commit code that hasn't passed all 3 checks - Fix ALL clippy warnings before moving on (zero tolerance) - If build fails, fix it immediately before continuing to next task -- Pre-commit hook will auto-enforce this (see `.claude/hooks/bash/pre-commit-format.sh`) - -**Why**: RTK is a production CLI tool used by developers in their workflows. Bugs break developer productivity. Quality gates prevent regressions and maintain user trust. **Performance verification** (for filter changes): - ```bash -# Benchmark before/after -hyperfine 'rtk git log -10' --warmup 3 +hyperfine 'rtk git log -10' --warmup 3 # before cargo build --release -hyperfine 'target/release/rtk git log -10' --warmup 3 - -# Memory profiling -/usr/bin/time -l target/release/rtk git status # macOS -/usr/bin/time -v target/release/rtk git status # Linux +hyperfine 'target/release/rtk git log -10' --warmup 3 # after (should be <10ms) ``` -## Testing Policy - -**Manual testing is REQUIRED** for filter changes and new commands: - -- **For new filters**: Test with real command (`rtk `), verify output matches expectations - - Example: `rtk git log -10` → inspect output, verify condensed correctly - - Example: `rtk cargo test` → verify only failures shown, not full output - -- **For hook changes**: Test in real Claude Code session, verify command rewriting works - - Create test Claude Code session - - Type raw command (e.g., `git status`) - - Verify hook rewrites to `rtk git status` - -- **For performance**: Run `hyperfine` comparison (before/after), verify <10ms startup - - Benchmark baseline: `hyperfine 'rtk git status' --warmup 3` - - Make changes, rebuild - - Benchmark again: `hyperfine 'target/release/rtk git status' --warmup 3` - - Compare results: startup time should be <10ms - -- **For cross-platform**: Test on macOS + Linux (Docker) + Windows (CI), verify shell escaping - - macOS (zsh): Test locally - - Linux (bash): Use Docker `docker run --rm -v $(pwd):/rtk -w /rtk rust:latest cargo test` - - Windows (PowerShell): Trust CI/CD pipeline or test manually if available - -**Anti-pattern**: Running only automated tests (`cargo test`, `cargo clippy`) without actually executing `rtk ` and inspecting output. - -**Example**: If fixing the `git log` filter, run `rtk git log -10` and verify: -1. Output is condensed (shorter than raw `git log -10`) -2. Critical info preserved (commit hashes, messages) -3. Format is readable and consistent -4. Exit code matches git's exit code (0 for success) - ## Working Directory Confirmation **ALWAYS confirm working directory before starting any work**: @@ -553,65 +167,3 @@ When user provides a numbered plan (QW1-QW4, Phase 1-5, sprint tasks, etc.): 3. **Never skip or reorder**: If a step is blocked, report it and ask before proceeding 4. **Track progress**: Use task list (TaskCreate/TaskUpdate) for plans with 3+ steps 5. **Validate assumptions**: Before starting, verify all referenced file paths exist and working directory is correct - -**Why**: Plan-driven execution produces better outcomes than ad-hoc implementation. Structured plans help maintain focus and prevent scope creep. - - -## Filter Development Checklist - -When adding a new filter (e.g., `rtk newcmd`): - -### Implementation -- [ ] Create filter module in `src/_cmd.rs` (or extend existing) -- [ ] Add `lazy_static!` regex patterns for parsing (compile once, reuse) -- [ ] Implement fallback to raw command on error (graceful degradation) -- [ ] Preserve exit codes (`std::process::exit(code)` if non-zero) - -### Testing -- [ ] Write snapshot test with real command output fixture (`tests/fixtures/_raw.txt`) -- [ ] Verify token savings ≥60% with `count_tokens()` assertion -- [ ] Test cross-platform shell escaping (macOS, Linux, Windows) -- [ ] Write unit tests for edge cases (empty output, errors, unicode, ANSI codes) - -### Integration -- [ ] Register filter in main.rs Commands enum -- [ ] Update README.md with new command support and token savings % -- [ ] Update CHANGELOG.md with feature description - -### Quality Gates -- [ ] Run `cargo fmt --all && cargo clippy --all-targets && cargo test` -- [ ] Benchmark startup time with `hyperfine` (verify <10ms) -- [ ] Test manually: `rtk ` and inspect output for correctness -- [ ] Verify fallback: Break filter intentionally, confirm raw command executes - -### Documentation -- [ ] Add command to this CLAUDE.md Module Responsibilities table -- [ ] Document token savings % (from tests) -- [ ] Add usage examples to README.md - -**Example workflow** (adding `rtk newcmd`): - -```bash -# 1. Create module -touch src/newcmd_cmd.rs - -# 2. Write test first (TDD) -echo 'raw command output fixture' > tests/fixtures/newcmd_raw.txt -# Add test in src/newcmd_cmd.rs - -# 3. Implement filter -# Add lazy_static regex, implement logic, add fallback - -# 4. Quality checks -cargo fmt --all && cargo clippy --all-targets && cargo test - -# 5. Benchmark -hyperfine 'rtk newcmd args' - -# 6. Manual test -rtk newcmd args -# Inspect output, verify condensed - -# 7. Document -# Update README.md, CHANGELOG.md, this file -``` diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 3221a21b..479c0ff5 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -27,6 +27,90 @@ | **Document** | Improve docs, add usage examples, clarify existing docs | --- +## Design Philosophy + +Four principles guide every RTK design decision. Understanding them helps you write contributions that fit naturally into the project. + +### Correctness VS Token Savings + +When a user or LLM explicitly requests detailed output via flags (e.g., `git log --comments`, `cargo test -- --nocapture`, `ls -la`), respect that intent. Compressing explicitly-requested detail defeats the purpose — the LLM asked for it because it needs it. + +Filters should be flag-aware: default output (no flags) gets aggressively compressed, but verbose/detailed flags should pass through more content. When in doubt, preserve correctness. + +> Example: `rtk cargo test` shows failures only (90% savings). But `rtk cargo test -- --nocapture` preserves all output because the user explicitly asked for it. + +### Transparency + +The LLM doesn't know RTK is involved — hooks rewrite commands silently. RTK's output must be a valid, useful subset of the original tool's output, not a different format the LLM wouldn't expect. If an LLM parses `git diff` output, RTK's filtered version must still look like `git diff` output. + +Don't invent new output formats. Don't add RTK-specific headers or markers in the default output. The filtered output should be indistinguishable from "a shorter version of the real command." + +### Never Block + +If a filter fails, fall back to raw output. RTK should never prevent a command from executing or producing output. Better to pass through unfiltered than to error out. Same for hooks: exit 0 on all error paths so the agent's command runs unmodified. + +Every filter needs a fallback path. Every hook must handle malformed input gracefully. + +### Zero Overhead + +<10ms startup. No async runtime. No config file I/O on the critical path. If developers perceive any delay, they'll disable RTK. Speed is the difference between adoption and abandonment. + +`lazy_static!` for all regex. No network calls. No disk reads in the hot path. Benchmark before/after with `hyperfine`. + +--- + +## What Belongs in RTK? + +RTK filters **development CLI commands** consumed by LLM coding assistants — the commands an AI agent runs during a coding session: test runners, linters, build tools, VCS operations, package managers, file operations. + +### In Scope + +Commands that produce **text output** (typically 100+ tokens) and can be compressed **60%+** without losing essential information for the LLM. + +- Test runners (vitest, pytest, cargo test, go test) +- Linters and type checkers (eslint, ruff, tsc, mypy) +- Build tools (cargo build, dotnet build, make, next build) +- VCS operations (git status/log/diff, gh pr/issue) +- Package managers (pnpm, pip, cargo install, brew) +- File operations (ls, tree, grep, find, cat/head/tail) +- Infrastructure tools with text output (docker, kubectl, terraform) + +### Out of Scope + +- Interactive TUIs (htop, vim, less) — not batch-mode compatible +- Binary output (images, compiled artifacts) — no text to filter +- Trivial commands (<100 tokens typical output) — not worth the overhead +- Commands with no text output — nothing to compress + +### TOML vs Rust: Which One? + +| Use **TOML filter** when | Use **Rust module** when | +|--------------------------|--------------------------| +| Output is plain text with predictable line structure | Output is structured (JSON, NDJSON) | +| Regex line filtering achieves 60%+ savings | Needs state machine parsing (e.g., pytest phases) | +| No need to inject CLI flags | Needs to inject flags like `--format json` | +| No cross-command routing | Routes to other commands (lint → ruff/mypy) | +| Examples: brew, df, shellcheck, rsync, ping | Examples: vitest, pytest, golangci-lint, gh | + +See [`src/filters/README.md`](src/filters/README.md) for TOML filter guidance and [`src/cmds/README.md`](src/cmds/README.md) for Rust module guidance. + +### Complete Contribution Checklist + +Adding a new filter or command requires changes in multiple places: + +1. **Create the filter** — TOML file in `src/filters/` or Rust module in `src/cmds//` +2. **Add rewrite pattern** — Entry in `src/discover/rules.rs` (PATTERNS + RULES arrays at matching index) so hooks auto-rewrite the command +3. **Register in main.rs** — (Rust modules only) Three changes: + - Add `pub mod mymod;` to the ecosystem's `mod.rs` (e.g., `src/cmds/system/mod.rs`) + - Add variant to `Commands` enum in `main.rs` with `#[arg(trailing_var_arg = true, allow_hyphen_values = true)]` + - Add routing match arm in `main.rs` to call `mymod::run()` +4. **Write tests** — Real fixture, snapshot test, token savings >= 60% +5. **Update docs** — README.md command list, CHANGELOG.md + +See [src/cmds/README.md](src/cmds/README.md#common-pattern) for the standard module template with timer, fallback, tee, and tracking. + +--- + ## Branch Naming Convention Every branch **must** follow one of these prefixes to identify the level of change: @@ -135,7 +219,7 @@ Every change **must** include tests. We follow **TDD (Red-Green-Refactor)**: wri ### How to Write Tests -Tests for new commands live **in the module file itself** inside a `#[cfg(test)] mod tests` block (e.g. tests for `src/kubectl_cmd.rs` go at the bottom of that same file). +Tests for new commands live **in the module file itself** inside a `#[cfg(test)] mod tests` block (e.g. tests for `src/cmds/cloud/container.rs` go at the bottom of that same file). **1. Create a fixture from real command output** (not synthetic data): ```bash diff --git a/docs/TECHNICAL.md b/docs/TECHNICAL.md index 9588f018..18e33b71 100644 --- a/docs/TECHNICAL.md +++ b/docs/TECHNICAL.md @@ -1,6 +1,8 @@ # RTK Technical Documentation -> For contributors and maintainers. See `CLAUDE.md` for development commands and coding guidelines. +> **Start here** for a guided tour of how RTK works end-to-end. For the deep reference (filtering taxonomy, performance benchmarks, architecture decisions), see [ARCHITECTURE.md](../ARCHITECTURE.md). +> +> See `CLAUDE.md` for development commands and coding guidelines. > Each folder has its own README.md with implementation details, file descriptions, and contribution guidelines. --- @@ -171,40 +173,34 @@ Tee is configurable (enabled/disabled, min size, max files, max file size) and n ## 4. Folder Map -``` -src/ -+-- main.rs # CLI entry point, Commands enum, command routing -+-- core/ # Shared infrastructure (8 files) -| +-- README.md # -> Details: tracking, config, tee, TOML filters -+-- hooks/ # Hook system (8 files) -| +-- README.md # -> Details: init, integrity, rewrite, verify -+-- analytics/ # Token savings analytics (4 files) -| +-- README.md # -> Details: gain, economics, session -+-- cmds/ # Command filter modules (45 files) -| +-- README.md # -> Details: common pattern, ecosystem organization -| +-- git/ # Git + GitHub CLI + Graphite (4 files) -| +-- rust/ # Cargo + runner (2 files) -| +-- js/ # JS/TS/Node ecosystem (9 files) -| +-- python/ # Python ecosystem (4 files) -| +-- go/ # Go ecosystem (2 files) -| +-- dotnet/ # .NET ecosystem (4 files) -| +-- cloud/ # Cloud and infra (5 files) -| +-- system/ # System utilities (13 files) -+-- discover/ # Claude Code history analysis -+-- learn/ # CLI correction detection -+-- parser/ # Parser infrastructure -+-- filters/ # 60+ TOML filter configs - -hooks/ # LLM agent hook scripts (root directory) -+-- README.md # -> Details: agent JSON formats, rewrite flow -+-- claude/ # Claude Code (shell hook) -+-- copilot/ # GitHub Copilot (Rust binary hook) -+-- cursor/ # Cursor IDE (shell hook) -+-- cline/ # Cline / Roo Code (rules file) -+-- windsurf/ # Windsurf / Cascade (rules file) -+-- codex/ # OpenAI Codex CLI (awareness doc) -+-- opencode/ # OpenCode (TypeScript plugin) -``` +Start here, then drill down into each README for file-level details. + +### `src/` — Rust source code + +| Directory | What it does | What you'll find in its README | +|-----------|-------------|-------------------------------| +| `main.rs` | CLI entry point, `Commands` enum, routing match | _(no README — read the file directly)_ | +| [`core/`](../src/core/README.md) | Shared infrastructure (8 files) | Tracking DB schema, config system, tee recovery, TOML filter engine, utility functions | +| [`hooks/`](../src/hooks/README.md) | Hook system (8 files) | Installation flow (`rtk init`), integrity verification, rewrite command, trust model | +| [`analytics/`](../src/analytics/README.md) | Token savings analytics (4 files) | `rtk gain` dashboard, Claude Code economics, ccusage parsing | +| [`cmds/`](../src/cmds/README.md) | **Command filters (45 files, 9 ecosystems)** | Common filter pattern, cross-command routing, token savings table, **links to each ecosystem** | +| [`discover/`](../src/discover/README.md) | History analysis + rewrite registry (5 files) | 70+ rewrite patterns, session providers, compound command splitting | +| [`learn/`](../src/learn/README.md) | CLI correction detection (3 files) | Error classification (6 types), correction pair detection, rule generation | +| [`parser/`](../src/parser/README.md) | Parser infrastructure (4 files) | Canonical types (TestResult, LintResult, etc.), 3-tier format modes, migration guide | +| [`filters/`](../src/filters/README.md) | 60+ TOML filter configs | TOML DSL syntax, 8-stage pipeline, inline testing, naming conventions | + +### `hooks/` — Deployed hook artifacts (root directory) + +| Directory | Agent | What you'll find in its README | +|-----------|-------|-------------------------------| +| [`hooks/`](../hooks/README.md) | _(parent)_ | **All JSON formats**, rewrite registry overview, exit code contract, override controls | +| [`claude/`](../hooks/claude/README.md) | Claude Code | Shell hook mechanism, `PreToolUse` JSON, test script | +| [`copilot/`](../hooks/copilot/README.md) | GitHub Copilot | Rust binary hook, VS Code Chat vs Copilot CLI dual format | +| [`cursor/`](../hooks/cursor/README.md) | Cursor IDE | Shell hook, empty JSON response requirement | +| [`cline/`](../hooks/cline/README.md) | Cline / Roo Code | Rules file (prompt-level, no programmatic hook) | +| [`windsurf/`](../hooks/windsurf/README.md) | Windsurf / Cascade | Rules file (workspace-scoped) | +| [`codex/`](../hooks/codex/README.md) | OpenAI Codex CLI | Awareness document, AGENTS.md integration | +| [`opencode/`](../hooks/opencode/README.md) | OpenCode | TypeScript plugin, zx library, in-place mutation | --- diff --git a/hooks/README.md b/hooks/README.md index a0b0265c..72ee98c6 100644 --- a/hooks/README.md +++ b/hooks/README.md @@ -31,16 +31,15 @@ All rewrite logic lives in the Rust binary (`src/discover/registry.rs`). Hook sc ## Directory Structure -``` -hooks/ - claude/ Claude Code (shell hook + settings.json) - copilot/ GitHub Copilot (VS Code Chat + Copilot CLI) - cursor/ Cursor IDE (shell hook) - cline/ Cline / Roo Code VS Code extension (rules file) - windsurf/ Windsurf / Cascade IDE (rules file) - codex/ OpenAI Codex CLI (awareness file) - opencode/ OpenCode (TypeScript plugin) -``` +Each agent subdirectory has its own README with hook-specific details: + +- **[`claude/`](claude/README.md)** — Shell hook, `PreToolUse` JSON format, `settings.json` patching, test script +- **[`copilot/`](copilot/README.md)** — Rust binary hook, dual format (VS Code Chat vs Copilot CLI), deny-with-suggestion fallback +- **[`cursor/`](cursor/README.md)** — Shell hook, Cursor JSON format, empty `{}` response requirement +- **[`cline/`](cline/README.md)** — Rules file (prompt-level), `.clinerules` project-local installation +- **[`windsurf/`](windsurf/README.md)** — Rules file (prompt-level), `.windsurfrules` workspace-scoped +- **[`codex/`](codex/README.md)** — Awareness document, `AGENTS.md` integration, `~/.codex/` location +- **[`opencode/`](opencode/README.md)** — TypeScript plugin, `zx` library, `tool.execute.before` event, in-place mutation ## Supported Agents @@ -209,3 +208,28 @@ Hooks are **non-blocking** -- they never prevent a command from executing: - `rtk rewrite` crashes: hook exits 0 (subprocess error ignored) - Filter logic error: fallback to raw command output +## Adding a New Agent Integration + +New integrations must follow the [Exit Code Contract](#exit-code-contract) and [Graceful Degradation](#graceful-degradation) above, as well as the project's [Design Philosophy](../CONTRIBUTING.md#design-philosophy). + +### Integration Tiers + +| Tier | Mechanism | Maintenance | Examples | +|------|-----------|-------------|----------| +| **Full hook** | Shell script or Rust binary, intercepts commands via agent's hook API | High — must track agent API changes | Claude Code, Cursor, Copilot, Gemini | +| **Plugin** | TypeScript/JS plugin in agent's plugin system | Medium — agent manages loading | OpenCode | +| **Rules file** | Prompt-level instructions the agent reads | Low — no code to break | Cline, Windsurf, Codex | + +### Eligibility + +RTK supports AI coding assistants that developers actually use day-to-day. To add a new agent: + +- Agent has a **documented, stable hook/plugin API** (not experimental/alpha) +- Agent is **actively maintained** (commit activity in last 3 months) +- Integration follows the **exit code contract** (exit 0 on all error paths) +- Hook output matches the **agent's expected JSON format** exactly + +### Maintenance + +If an agent's API changes and the hook breaks, the integration should be updated promptly. If the agent becomes unmaintained or the hook can't be fixed, the integration may be deprecated with a release note. + diff --git a/hooks/claude/README.md b/hooks/claude/README.md index b0cd8f1e..ba2dea1a 100644 --- a/hooks/claude/README.md +++ b/hooks/claude/README.md @@ -1,5 +1,7 @@ # Claude Code Hooks +> Part of [`hooks/`](../README.md) — see also [`src/hooks/`](../../src/hooks/README.md) for installation code + ## Specifics - Shell-based `PreToolUse` hook -- requires `jq` for JSON parsing diff --git a/hooks/rtk-awareness.md b/hooks/claude/rtk-awareness.md similarity index 100% rename from hooks/rtk-awareness.md rename to hooks/claude/rtk-awareness.md diff --git a/hooks/rtk-rewrite.sh b/hooks/claude/rtk-rewrite.sh similarity index 100% rename from hooks/rtk-rewrite.sh rename to hooks/claude/rtk-rewrite.sh diff --git a/hooks/test-rtk-rewrite.sh b/hooks/claude/test-rtk-rewrite.sh old mode 100755 new mode 100644 similarity index 100% rename from hooks/test-rtk-rewrite.sh rename to hooks/claude/test-rtk-rewrite.sh diff --git a/hooks/cline/README.md b/hooks/cline/README.md index 8f11bc3f..134e7a64 100644 --- a/hooks/cline/README.md +++ b/hooks/cline/README.md @@ -1,5 +1,7 @@ # Cline / Roo Code Hooks +> Part of [`hooks/`](../README.md) — see also [`src/hooks/`](../../src/hooks/README.md) for installation code + ## Specifics - Prompt-level guidance only (no programmatic hook) -- relies on Cline reading custom instructions diff --git a/hooks/cline-rtk-rules.md b/hooks/cline/rules.md similarity index 100% rename from hooks/cline-rtk-rules.md rename to hooks/cline/rules.md diff --git a/hooks/codex/README.md b/hooks/codex/README.md index 9c8362fb..e922e636 100644 --- a/hooks/codex/README.md +++ b/hooks/codex/README.md @@ -1,5 +1,7 @@ # Codex CLI Hooks +> Part of [`hooks/`](../README.md) — see also [`src/hooks/`](../../src/hooks/README.md) for installation code + ## Specifics - Prompt-level guidance via awareness document -- no programmatic hook diff --git a/hooks/rtk-awareness-codex.md b/hooks/codex/rtk-awareness.md similarity index 100% rename from hooks/rtk-awareness-codex.md rename to hooks/codex/rtk-awareness.md diff --git a/hooks/copilot/README.md b/hooks/copilot/README.md index 16500363..5f6097c6 100644 --- a/hooks/copilot/README.md +++ b/hooks/copilot/README.md @@ -1,5 +1,7 @@ # GitHub Copilot Hooks +> Part of [`hooks/`](../README.md) — see also [`src/hooks/`](../../src/hooks/README.md) for installation code + ## Specifics - Uses the `rtk hook copilot` Rust binary (not a shell script) -- no `jq` dependency diff --git a/hooks/copilot-rtk-awareness.md b/hooks/copilot/rtk-awareness.md similarity index 100% rename from hooks/copilot-rtk-awareness.md rename to hooks/copilot/rtk-awareness.md diff --git a/hooks/test-copilot-rtk-rewrite.sh b/hooks/copilot/test-rtk-rewrite.sh old mode 100755 new mode 100644 similarity index 100% rename from hooks/test-copilot-rtk-rewrite.sh rename to hooks/copilot/test-rtk-rewrite.sh diff --git a/hooks/cursor/README.md b/hooks/cursor/README.md index 467c3b7b..83e7f027 100644 --- a/hooks/cursor/README.md +++ b/hooks/cursor/README.md @@ -1,5 +1,7 @@ # Cursor IDE Hooks +> Part of [`hooks/`](../README.md) — see also [`src/hooks/`](../../src/hooks/README.md) for installation code + ## Specifics - Same delegating pattern as Claude Code hook but outputs Cursor's JSON format (`permission`/`updated_input` instead of `hookSpecificOutput`/`updatedInput`) diff --git a/hooks/cursor-rtk-rewrite.sh b/hooks/cursor/rtk-rewrite.sh old mode 100755 new mode 100644 similarity index 100% rename from hooks/cursor-rtk-rewrite.sh rename to hooks/cursor/rtk-rewrite.sh diff --git a/hooks/opencode/README.md b/hooks/opencode/README.md index 94bca2af..8edc93cc 100644 --- a/hooks/opencode/README.md +++ b/hooks/opencode/README.md @@ -1,5 +1,7 @@ # OpenCode Hooks +> Part of [`hooks/`](../README.md) — see also [`src/hooks/`](../../src/hooks/README.md) for installation code + ## Specifics - TypeScript plugin using the zx library (not a shell hook) diff --git a/hooks/opencode-rtk.ts b/hooks/opencode/rtk.ts similarity index 100% rename from hooks/opencode-rtk.ts rename to hooks/opencode/rtk.ts diff --git a/hooks/windsurf/README.md b/hooks/windsurf/README.md index c4a3653b..4cb5da92 100644 --- a/hooks/windsurf/README.md +++ b/hooks/windsurf/README.md @@ -1,5 +1,7 @@ # Windsurf (Cascade) Hooks +> Part of [`hooks/`](../README.md) — see also [`src/hooks/`](../../src/hooks/README.md) for installation code + ## Specifics - Prompt-level guidance only (no programmatic hook) -- relies on Windsurf Cascade reading rules files diff --git a/hooks/windsurf-rtk-rules.md b/hooks/windsurf/rules.md similarity index 100% rename from hooks/windsurf-rtk-rules.md rename to hooks/windsurf/rules.md diff --git a/src/analytics/README.md b/src/analytics/README.md index b75c939a..584b52d4 100644 --- a/src/analytics/README.md +++ b/src/analytics/README.md @@ -1,5 +1,7 @@ # Analytics +> See also [docs/TECHNICAL.md](../../docs/TECHNICAL.md) for the full architecture overview + ## Scope **Read-only dashboards** over the tracking database. Analytics presents the value that `cmds/` creates — it queries token savings, correlates with external spending data, and surfaces adoption opportunities. It never modifies the tracking DB. @@ -15,15 +17,5 @@ Token savings analytics, economic modeling, and adoption metrics. These modules read from the SQLite tracking database to produce dashboards, spending estimates, and session-level adoption reports that help users understand the value RTK provides. - - -## Files -| File | Responsibility | -|------|---------------| -| gain.rs | `rtk gain` command -- token savings dashboard with ASCII graphs, per-command history, quota consumption estimates, and cumulative savings over time; the primary user-facing analytics view | -| cc_economics.rs | `rtk cc-economics` command -- combines Claude Code API spending data with RTK savings to show net cost reduction; correlates token savings with dollar amounts | -| ccusage.rs | Claude Code usage data parser; defines `CcusagePeriod` and `Granularity` types; reads Claude Code session spending records to feed into `cc_economics.rs` | -| session_cmd.rs | `rtk session` command -- per-session RTK adoption analysis; shows which commands in a coding session were routed through RTK vs executed raw, highlighting missed savings opportunities | - ## Adding New Functionality To add a new analytics view: (1) create a new `*_cmd.rs` file in this directory, (2) query `core/tracking` for the metrics you need using the existing `TrackingDb` API, (3) register the command in `main.rs` under the `Commands` enum, and (4) add `#[cfg(test)]` unit tests with sample tracking data. Analytics modules should be read-only against the tracking database and never modify it. diff --git a/src/cc_economics.rs b/src/analytics/cc_economics.rs similarity index 99% rename from src/cc_economics.rs rename to src/analytics/cc_economics.rs index 6f50f677..693dc61e 100644 --- a/src/cc_economics.rs +++ b/src/analytics/cc_economics.rs @@ -8,9 +8,9 @@ use chrono::NaiveDate; use serde::Serialize; use std::collections::HashMap; -use crate::ccusage::{self, CcusagePeriod, Granularity}; -use crate::tracking::{DayStats, MonthStats, Tracker, WeekStats}; -use crate::utils::{format_cpt, format_tokens, format_usd}; +use super::ccusage::{self, CcusagePeriod, Granularity}; +use crate::core::tracking::{DayStats, MonthStats, Tracker, WeekStats}; +use crate::core::utils::{format_cpt, format_tokens, format_usd}; // ── Constants ── @@ -207,7 +207,7 @@ fn merge_daily(cc: Option>, rtk: Vec) -> Vec>, rtk: Vec) -> Vec>, rtk: Vec) -> Vec Result<()> { - // 1. Start timer for tracking let timer = tracking::TimedExecution::start(); + let output = resolved_command("mycmd").args(&args).output().context("Failed to execute mycmd")?; + let raw = format!("{}\n{}", String::from_utf8_lossy(&output.stdout), String::from_utf8_lossy(&output.stderr)); - // 2. Execute the underlying command - let output = resolved_command("mycmd") - .args(&args.to_cmd_args()) - .output() - .context("Failed to execute mycmd")?; + let filtered = filter_output(&raw).unwrap_or_else(|e| { + eprintln!("rtk: filter warning: {}", e); + raw.clone() // Fallback to raw on filter failure + }); - let stdout = String::from_utf8_lossy(&output.stdout); - let stderr = String::from_utf8_lossy(&output.stderr); - let raw = format!("{}\n{}", stdout, stderr); - - // 3. Filter the output (with fallback to raw on error) - let filtered = filter_output(&stdout) - .unwrap_or_else(|e| { - eprintln!("rtk: filter warning: {}", e); - raw.clone() // Passthrough on failure - }); - - // 4. Tee raw output on failure (for LLM re-read) let exit_code = output.status.code().unwrap_or(1); if let Some(hint) = tee::tee_and_hint(&raw, "mycmd", exit_code) { println!("{}\n{}", filtered, hint); @@ -61,23 +57,13 @@ pub fn run(args: MyArgs, verbose: u8) -> Result<()> { println!("{}", filtered); } - // 5. Track token savings to SQLite timer.track("mycmd args", "rtk mycmd args", &raw, &filtered); - - // 6. Propagate exit code - if !output.status.success() { - std::process::exit(exit_code); - } + if !output.status.success() { std::process::exit(exit_code); } Ok(()) } ``` -Key aspects of this pattern: -- **`TimedExecution`**: Records elapsed time, estimates tokens (`ceil(chars / 4.0)`), writes to SQLite -- **`resolved_command()`**: Finds the command in PATH, handles aliases -- **Fallback**: Filter errors fall back to raw output (never block the user) -- **Tee recovery**: On failure, saves raw output to disk with a hint line for LLM re-read -- **Exit code**: Always propagated via `std::process::exit(code)` for CI/CD reliability +Six phases: **timer** → **execute** → **filter (with fallback)** → **tee on failure** → **track** → **exit code**. See [core/README.md](../core/README.md#consumer-contracts) for the contracts each phase must honor. ## Token Savings by Category @@ -154,12 +140,4 @@ All modules accept `verbose: u8`. Use it to print debug info (command being run, ## Adding a New Command Filter -1. Create `_cmd.rs` in the appropriate ecosystem subdirectory -2. Follow the common pattern above (timer, execute, filter with fallback, tee, track, exit code) -3. Use `lazy_static!` for all regex patterns -4. Add the command to the `Commands` enum in `main.rs` -5. Create a test fixture in `tests/fixtures/` from real command output -6. Write snapshot test (`assert_snapshot!`) and token savings test (verify >= 60% reduction) -7. Run `cargo fmt --all && cargo clippy --all-targets && cargo test` - -See the Filter Development Checklist in `CLAUDE.md` and `.claude/rules/rust-patterns.md` for the full module structure. +Follow the [Common Pattern](#common-pattern) above (timer, execute, filter with fallback, tee, track, exit code). For the full step-by-step checklist, see [CONTRIBUTING.md](../../CONTRIBUTING.md#complete-contribution-checklist). For the Rust module structure, see [`.claude/rules/rust-patterns.md`](../../.claude/rules/rust-patterns.md). diff --git a/src/cmds/cloud/README.md b/src/cmds/cloud/README.md index 5459b16e..7bfa9228 100644 --- a/src/cmds/cloud/README.md +++ b/src/cmds/cloud/README.md @@ -1,5 +1,7 @@ # Cloud and Infrastructure +> Part of [`src/cmds/`](../README.md) — see also [docs/TECHNICAL.md](../../../docs/TECHNICAL.md) + ## Specifics - `aws_cmd.rs` forces `--output json` for structured parsing diff --git a/src/aws_cmd.rs b/src/cmds/cloud/aws_cmd.rs similarity index 99% rename from src/aws_cmd.rs rename to src/cmds/cloud/aws_cmd.rs index c070ef54..8b701f26 100644 --- a/src/aws_cmd.rs +++ b/src/cmds/cloud/aws_cmd.rs @@ -4,8 +4,8 @@ //! Specialized filters for high-frequency commands (STS, S3, EC2, ECS, RDS, CloudFormation). use crate::json_cmd; -use crate::tracking; -use crate::utils::{join_with_overflow, resolved_command, truncate_iso_date}; +use crate::core::tracking; +use crate::core::utils::{join_with_overflow, resolved_command, truncate_iso_date}; use anyhow::{Context, Result}; use serde_json::Value; diff --git a/src/container.rs b/src/cmds/cloud/container.rs similarity index 99% rename from src/container.rs rename to src/cmds/cloud/container.rs index e609de0c..b4e0057a 100644 --- a/src/container.rs +++ b/src/cmds/cloud/container.rs @@ -1,5 +1,7 @@ -use crate::tracking; -use crate::utils::resolved_command; +//! Filters Docker and kubectl output into compact summaries. + +use crate::core::tracking; +use crate::core::utils::resolved_command; use anyhow::{Context, Result}; use std::ffi::OsString; diff --git a/src/curl_cmd.rs b/src/cmds/cloud/curl_cmd.rs similarity index 96% rename from src/curl_cmd.rs rename to src/cmds/cloud/curl_cmd.rs index 90be2236..046ae0da 100644 --- a/src/curl_cmd.rs +++ b/src/cmds/cloud/curl_cmd.rs @@ -1,6 +1,8 @@ +//! Runs curl and auto-compresses JSON responses. + use crate::json_cmd; -use crate::tracking; -use crate::utils::{resolved_command, truncate}; +use crate::core::tracking; +use crate::core::utils::{resolved_command, truncate}; use anyhow::{Context, Result}; pub fn run(args: &[String], verbose: u8) -> Result<()> { diff --git a/src/cmds/cloud/mod.rs b/src/cmds/cloud/mod.rs new file mode 100644 index 00000000..8917a6c2 --- /dev/null +++ b/src/cmds/cloud/mod.rs @@ -0,0 +1,7 @@ +//! Cloud and infrastructure tool filters. + +pub mod aws_cmd; +pub mod container; +pub mod curl_cmd; +pub mod psql_cmd; +pub mod wget_cmd; diff --git a/src/psql_cmd.rs b/src/cmds/cloud/psql_cmd.rs similarity index 98% rename from src/psql_cmd.rs rename to src/cmds/cloud/psql_cmd.rs index 430e958d..9ec243be 100644 --- a/src/psql_cmd.rs +++ b/src/cmds/cloud/psql_cmd.rs @@ -3,8 +3,8 @@ //! Detects table and expanded display formats, strips borders/padding, //! and produces compact tab-separated or key=value output. -use crate::tracking; -use crate::utils::resolved_command; +use crate::core::tracking; +use crate::core::utils::resolved_command; use anyhow::{Context, Result}; use lazy_static::lazy_static; use regex::Regex; @@ -49,7 +49,7 @@ pub fn run(args: &[String], verbose: u8) -> Result<()> { let filtered = filter_psql_output(&stdout); - if let Some(hint) = crate::tee::tee_and_hint(&stdout, "psql", exit_code) { + if let Some(hint) = crate::core::tee::tee_and_hint(&stdout, "psql", exit_code) { println!("{}\n{}", filtered, hint); } else { println!("{}", filtered); diff --git a/src/wget_cmd.rs b/src/cmds/cloud/wget_cmd.rs similarity index 99% rename from src/wget_cmd.rs rename to src/cmds/cloud/wget_cmd.rs index 722f88f1..32996ac3 100644 --- a/src/wget_cmd.rs +++ b/src/cmds/cloud/wget_cmd.rs @@ -1,5 +1,5 @@ -use crate::tracking; -use crate::utils::resolved_command; +use crate::core::tracking; +use crate::core::utils::resolved_command; use anyhow::{Context, Result}; /// Compact wget - strips progress bars, shows only result diff --git a/src/cmds/dotnet/README.md b/src/cmds/dotnet/README.md index f32eb384..707aaab5 100644 --- a/src/cmds/dotnet/README.md +++ b/src/cmds/dotnet/README.md @@ -1,5 +1,7 @@ # .NET Ecosystem +> Part of [`src/cmds/`](../README.md) — see also [docs/TECHNICAL.md](../../../docs/TECHNICAL.md) + ## Specifics - `dotnet_cmd.rs` uses `DotnetCommands` sub-enum in main.rs diff --git a/src/binlog.rs b/src/cmds/dotnet/binlog.rs similarity index 99% rename from src/binlog.rs rename to src/cmds/dotnet/binlog.rs index 8146cb5a..027f482f 100644 --- a/src/binlog.rs +++ b/src/cmds/dotnet/binlog.rs @@ -1,4 +1,6 @@ -use crate::utils::strip_ansi; +//! Reads MSBuild binary log files and extracts errors and test results. + +use crate::core::utils::strip_ansi; use anyhow::{Context, Result}; use flate2::read::GzDecoder; use lazy_static::lazy_static; @@ -1535,7 +1537,7 @@ Restore failed with 1 error(s) in 1.0s #[test] fn test_parse_build_from_fixture_text() { - let input = include_str!("../tests/fixtures/dotnet/build_failed.txt"); + let input = include_str!("../../../tests/fixtures/dotnet/build_failed.txt"); let summary = parse_build_from_text(input); assert_eq!(summary.errors.len(), 1); @@ -1571,7 +1573,7 @@ Time Elapsed 00:00:00.12 #[test] fn test_parse_test_from_fixture_text() { - let input = include_str!("../tests/fixtures/dotnet/test_failed.txt"); + let input = include_str!("../../../tests/fixtures/dotnet/test_failed.txt"); let summary = parse_test_from_text(input); assert_eq!(summary.failed, 1); diff --git a/src/dotnet_cmd.rs b/src/cmds/dotnet/dotnet_cmd.rs similarity index 99% rename from src/dotnet_cmd.rs rename to src/cmds/dotnet/dotnet_cmd.rs index dde3bba5..5f9f9b71 100644 --- a/src/dotnet_cmd.rs +++ b/src/cmds/dotnet/dotnet_cmd.rs @@ -1,8 +1,10 @@ +//! Filters dotnet CLI output — build, test, and format results. + use crate::binlog; use crate::dotnet_format_report; use crate::dotnet_trx; -use crate::tracking; -use crate::utils::{resolved_command, truncate}; +use crate::core::tracking; +use crate::core::utils::{resolved_command, truncate}; use anyhow::{Context, Result}; use quick_xml::events::Event; use quick_xml::Reader; diff --git a/src/dotnet_format_report.rs b/src/cmds/dotnet/dotnet_format_report.rs similarity index 98% rename from src/dotnet_format_report.rs rename to src/cmds/dotnet/dotnet_format_report.rs index 5b8837ff..c0223d8c 100644 --- a/src/dotnet_format_report.rs +++ b/src/cmds/dotnet/dotnet_format_report.rs @@ -1,3 +1,5 @@ +//! Parses dotnet format JSON reports into compact summaries. + use anyhow::{Context, Result}; use serde::Deserialize; use std::fs::File; diff --git a/src/dotnet_trx.rs b/src/cmds/dotnet/dotnet_trx.rs similarity index 99% rename from src/dotnet_trx.rs rename to src/cmds/dotnet/dotnet_trx.rs index 8c967666..ed372645 100644 --- a/src/dotnet_trx.rs +++ b/src/cmds/dotnet/dotnet_trx.rs @@ -1,3 +1,5 @@ +//! Parses .trx test result files (Visual Studio XML format) into compact summaries. + use crate::binlog::{FailedTest, TestSummary}; use chrono::{DateTime, FixedOffset}; use quick_xml::events::{BytesStart, Event}; diff --git a/src/cmds/dotnet/mod.rs b/src/cmds/dotnet/mod.rs new file mode 100644 index 00000000..ce6bfa48 --- /dev/null +++ b/src/cmds/dotnet/mod.rs @@ -0,0 +1,6 @@ +//! .NET ecosystem filters. + +pub mod binlog; +pub mod dotnet_cmd; +pub mod dotnet_format_report; +pub mod dotnet_trx; diff --git a/src/cmds/git/README.md b/src/cmds/git/README.md index bca9ef70..6b8dc570 100644 --- a/src/cmds/git/README.md +++ b/src/cmds/git/README.md @@ -1,5 +1,7 @@ # Git and VCS +> Part of [`src/cmds/`](../README.md) — see also [docs/TECHNICAL.md](../../../docs/TECHNICAL.md) + ## Specifics - **git.rs** uses `trailing_var_arg = true` + `allow_hyphen_values = true` so native git flags (`--oneline`, `--cached`, etc.) pass through correctly diff --git a/src/diff_cmd.rs b/src/cmds/git/diff_cmd.rs similarity index 98% rename from src/diff_cmd.rs rename to src/cmds/git/diff_cmd.rs index d9299eb5..6cb3bcf6 100644 --- a/src/diff_cmd.rs +++ b/src/cmds/git/diff_cmd.rs @@ -1,5 +1,7 @@ -use crate::tracking; -use crate::utils::truncate; +//! Compares two files and shows only the changed lines. + +use crate::core::tracking; +use crate::core::utils::truncate; use anyhow::Result; use std::fs; use std::path::Path; diff --git a/src/gh_cmd.rs b/src/cmds/git/gh_cmd.rs similarity index 99% rename from src/gh_cmd.rs rename to src/cmds/git/gh_cmd.rs index 2477bbd6..5847775b 100644 --- a/src/gh_cmd.rs +++ b/src/cmds/git/gh_cmd.rs @@ -4,8 +4,8 @@ //! Focuses on extracting essential information from JSON outputs. use crate::git; -use crate::tracking; -use crate::utils::{ok_confirmation, resolved_command, truncate}; +use crate::core::tracking; +use crate::core::utils::{ok_confirmation, resolved_command, truncate}; use anyhow::{Context, Result}; use lazy_static::lazy_static; use regex::Regex; diff --git a/src/git.rs b/src/cmds/git/git.rs similarity index 99% rename from src/git.rs rename to src/cmds/git/git.rs index 4bb7f674..9718486f 100644 --- a/src/git.rs +++ b/src/cmds/git/git.rs @@ -1,6 +1,8 @@ -use crate::config; -use crate::tracking; -use crate::utils::resolved_command; +//! Filters git output — log, status, diff, and more — keeping just the essential info. + +use crate::core::config; +use crate::core::tracking; +use crate::core::utils::resolved_command; use anyhow::{Context, Result}; use std::ffi::OsString; use std::process::Command; diff --git a/src/gt_cmd.rs b/src/cmds/git/gt_cmd.rs similarity index 98% rename from src/gt_cmd.rs rename to src/cmds/git/gt_cmd.rs index 88b38b91..580778ff 100644 --- a/src/gt_cmd.rs +++ b/src/cmds/git/gt_cmd.rs @@ -1,5 +1,7 @@ -use crate::tracking; -use crate::utils::{ok_confirmation, resolved_command, strip_ansi, truncate}; +//! Filters Graphite (gt) CLI output for stacking workflows. + +use crate::core::tracking; +use crate::core::utils::{ok_confirmation, resolved_command, strip_ansi, truncate}; use anyhow::{Context, Result}; use lazy_static::lazy_static; use regex::Regex; @@ -59,7 +61,7 @@ fn run_gt_filtered( filter_fn(&clean) }; - if let Some(hint) = crate::tee::tee_and_hint(&raw, tee_label, exit_code) { + if let Some(hint) = crate::core::tee::tee_and_hint(&raw, tee_label, exit_code) { println!("{}\n{}", output, hint); } else { println!("{}", output); diff --git a/src/cmds/git/mod.rs b/src/cmds/git/mod.rs new file mode 100644 index 00000000..a587ec11 --- /dev/null +++ b/src/cmds/git/mod.rs @@ -0,0 +1,6 @@ +//! Git ecosystem filters. + +pub mod diff_cmd; +pub mod gh_cmd; +pub mod git; +pub mod gt_cmd; diff --git a/src/cmds/go/README.md b/src/cmds/go/README.md index 95879efd..3f42ee9c 100644 --- a/src/cmds/go/README.md +++ b/src/cmds/go/README.md @@ -1,5 +1,7 @@ # Go Ecosystem +> Part of [`src/cmds/`](../README.md) — see also [docs/TECHNICAL.md](../../../docs/TECHNICAL.md) + ## Specifics - `go_cmd.rs` uses `GoCommands` sub-enum in main.rs (same pattern as git/cargo) diff --git a/src/go_cmd.rs b/src/cmds/go/go_cmd.rs similarity index 97% rename from src/go_cmd.rs rename to src/cmds/go/go_cmd.rs index d250c427..7530f489 100644 --- a/src/go_cmd.rs +++ b/src/cmds/go/go_cmd.rs @@ -1,5 +1,7 @@ -use crate::tracking; -use crate::utils::{resolved_command, truncate}; +//! Filters Go command output — test results, build errors, vet warnings. + +use crate::core::tracking; +use crate::core::utils::{resolved_command, truncate}; use anyhow::{Context, Result}; use serde::Deserialize; use std::collections::HashMap; @@ -69,7 +71,7 @@ pub fn run_test(args: &[String], verbose: u8) -> Result<()> { .unwrap_or(if output.status.success() { 0 } else { 1 }); let filtered = filter_go_test_json(&stdout); - if let Some(hint) = crate::tee::tee_and_hint(&raw, "go_test", exit_code) { + if let Some(hint) = crate::core::tee::tee_and_hint(&raw, "go_test", exit_code) { println!("{}\n{}", filtered, hint); } else { println!("{}", filtered); @@ -123,7 +125,7 @@ pub fn run_build(args: &[String], verbose: u8) -> Result<()> { .unwrap_or(if output.status.success() { 0 } else { 1 }); let filtered = filter_go_build(&raw); - if let Some(hint) = crate::tee::tee_and_hint(&raw, "go_build", exit_code) { + if let Some(hint) = crate::core::tee::tee_and_hint(&raw, "go_build", exit_code) { if !filtered.is_empty() { println!("{}\n{}", filtered, hint); } else { @@ -176,7 +178,7 @@ pub fn run_vet(args: &[String], verbose: u8) -> Result<()> { .unwrap_or(if output.status.success() { 0 } else { 1 }); let filtered = filter_go_vet(&raw); - if let Some(hint) = crate::tee::tee_and_hint(&raw, "go_vet", exit_code) { + if let Some(hint) = crate::core::tee::tee_and_hint(&raw, "go_vet", exit_code) { if !filtered.is_empty() { println!("{}\n{}", filtered, hint); } else { diff --git a/src/golangci_cmd.rs b/src/cmds/go/golangci_cmd.rs similarity index 98% rename from src/golangci_cmd.rs rename to src/cmds/go/golangci_cmd.rs index b2fdcd28..cb899ebd 100644 --- a/src/golangci_cmd.rs +++ b/src/cmds/go/golangci_cmd.rs @@ -1,6 +1,8 @@ -use crate::config; -use crate::tracking; -use crate::utils::{resolved_command, truncate}; +//! Filters golangci-lint output, grouping issues by rule. + +use crate::core::config; +use crate::core::tracking; +use crate::core::utils::{resolved_command, truncate}; use anyhow::{Context, Result}; use serde::Deserialize; use std::collections::HashMap; @@ -510,7 +512,7 @@ mod tests { #[test] fn test_golangci_v2_token_savings() { - let raw = include_str!("../tests/fixtures/golangci_v2_json.txt"); + let raw = include_str!("../../../tests/fixtures/golangci_v2_json.txt"); let filtered = filter_golangci_json(raw, 2); let savings = 100.0 - (count_tokens(&filtered) as f64 / count_tokens(raw) as f64 * 100.0); diff --git a/src/cmds/go/mod.rs b/src/cmds/go/mod.rs new file mode 100644 index 00000000..0892ab45 --- /dev/null +++ b/src/cmds/go/mod.rs @@ -0,0 +1,4 @@ +//! Go ecosystem filters. + +pub mod go_cmd; +pub mod golangci_cmd; diff --git a/src/cmds/js/README.md b/src/cmds/js/README.md index 097a26d3..49dfebc6 100644 --- a/src/cmds/js/README.md +++ b/src/cmds/js/README.md @@ -1,5 +1,7 @@ # JavaScript / TypeScript / Node +> Part of [`src/cmds/`](../README.md) — see also [docs/TECHNICAL.md](../../../docs/TECHNICAL.md) + ## Specifics - `utils::package_manager_exec()` auto-detects pnpm/yarn/npm -- JS modules should use this instead of hardcoding a package manager diff --git a/src/lint_cmd.rs b/src/cmds/js/lint_cmd.rs similarity index 98% rename from src/lint_cmd.rs rename to src/cmds/js/lint_cmd.rs index ca136e67..bb384bae 100644 --- a/src/lint_cmd.rs +++ b/src/cmds/js/lint_cmd.rs @@ -1,8 +1,10 @@ -use crate::config; +//! Filters ESLint and Biome linter output, grouping violations by rule. + +use crate::core::config; use crate::mypy_cmd; use crate::ruff_cmd; -use crate::tracking; -use crate::utils::{package_manager_exec, resolved_command, truncate}; +use crate::core::tracking; +use crate::core::utils::{package_manager_exec, resolved_command, truncate}; use anyhow::{Context, Result}; use serde::{Deserialize, Serialize}; use std::collections::HashMap; @@ -206,7 +208,7 @@ pub fn run(args: &[String], verbose: u8) -> Result<()> { .status .code() .unwrap_or(if output.status.success() { 0 } else { 1 }); - if let Some(hint) = crate::tee::tee_and_hint(&raw, "lint", exit_code) { + if let Some(hint) = crate::core::tee::tee_and_hint(&raw, "lint", exit_code) { println!("{}\n{}", filtered, hint); } else { println!("{}", filtered); diff --git a/src/cmds/js/mod.rs b/src/cmds/js/mod.rs new file mode 100644 index 00000000..0ddd3dad --- /dev/null +++ b/src/cmds/js/mod.rs @@ -0,0 +1,11 @@ +//! JavaScript and TypeScript ecosystem filters. + +pub mod lint_cmd; +pub mod next_cmd; +pub mod npm_cmd; +pub mod playwright_cmd; +pub mod pnpm_cmd; +pub mod prettier_cmd; +pub mod prisma_cmd; +pub mod tsc_cmd; +pub mod vitest_cmd; diff --git a/src/next_cmd.rs b/src/cmds/js/next_cmd.rs similarity index 97% rename from src/next_cmd.rs rename to src/cmds/js/next_cmd.rs index e958258d..5a7ad353 100644 --- a/src/next_cmd.rs +++ b/src/cmds/js/next_cmd.rs @@ -1,5 +1,7 @@ -use crate::tracking; -use crate::utils::{resolved_command, strip_ansi, tool_exists, truncate}; +//! Filters Next.js build output down to route metrics and bundle sizes. + +use crate::core::tracking; +use crate::core::utils::{resolved_command, strip_ansi, tool_exists, truncate}; use anyhow::{Context, Result}; use regex::Regex; diff --git a/src/npm_cmd.rs b/src/cmds/js/npm_cmd.rs similarity index 97% rename from src/npm_cmd.rs rename to src/cmds/js/npm_cmd.rs index 1d35d41f..7c86fe77 100644 --- a/src/npm_cmd.rs +++ b/src/cmds/js/npm_cmd.rs @@ -1,5 +1,7 @@ -use crate::tracking; -use crate::utils::resolved_command; +//! Filters npm output and auto-injects the "run" subcommand when appropriate. + +use crate::core::tracking; +use crate::core::utils::resolved_command; use anyhow::{Context, Result}; /// Known npm subcommands that should NOT get "run" injected. diff --git a/src/playwright_cmd.rs b/src/cmds/js/playwright_cmd.rs similarity index 98% rename from src/playwright_cmd.rs rename to src/cmds/js/playwright_cmd.rs index ce6f0fe7..a2a80542 100644 --- a/src/playwright_cmd.rs +++ b/src/cmds/js/playwright_cmd.rs @@ -1,5 +1,7 @@ -use crate::tracking; -use crate::utils::{detect_package_manager, resolved_command, strip_ansi}; +//! Filters Playwright E2E test output to show only failures. + +use crate::core::tracking; +use crate::core::utils::{detect_package_manager, resolved_command, strip_ansi}; use anyhow::{Context, Result}; use regex::Regex; use serde::Deserialize; @@ -315,7 +317,7 @@ pub fn run(args: &[String], verbose: u8) -> Result<()> { }; let exit_code = output.status.code().unwrap_or(1); - if let Some(hint) = crate::tee::tee_and_hint(&raw, "playwright", exit_code) { + if let Some(hint) = crate::core::tee::tee_and_hint(&raw, "playwright", exit_code) { println!("{}\n{}", filtered, hint); } else { println!("{}", filtered); diff --git a/src/pnpm_cmd.rs b/src/cmds/js/pnpm_cmd.rs similarity index 99% rename from src/pnpm_cmd.rs rename to src/cmds/js/pnpm_cmd.rs index 6690c16e..9746b38f 100644 --- a/src/pnpm_cmd.rs +++ b/src/cmds/js/pnpm_cmd.rs @@ -1,5 +1,7 @@ -use crate::tracking; -use crate::utils::resolved_command; +//! Filters pnpm output — dependency trees, install logs, outdated packages. + +use crate::core::tracking; +use crate::core::utils::resolved_command; use anyhow::{Context, Result}; use serde::Deserialize; use std::collections::HashMap; diff --git a/src/prettier_cmd.rs b/src/cmds/js/prettier_cmd.rs similarity index 97% rename from src/prettier_cmd.rs rename to src/cmds/js/prettier_cmd.rs index 2bbc98fe..af6c27a9 100644 --- a/src/prettier_cmd.rs +++ b/src/cmds/js/prettier_cmd.rs @@ -1,5 +1,7 @@ -use crate::tracking; -use crate::utils::package_manager_exec; +//! Filters Prettier output to show only files that need formatting. + +use crate::core::tracking; +use crate::core::utils::package_manager_exec; use anyhow::{Context, Result}; pub fn run(args: &[String], verbose: u8) -> Result<()> { diff --git a/src/prisma_cmd.rs b/src/cmds/js/prisma_cmd.rs similarity index 98% rename from src/prisma_cmd.rs rename to src/cmds/js/prisma_cmd.rs index a82ece07..a1299ede 100644 --- a/src/prisma_cmd.rs +++ b/src/cmds/js/prisma_cmd.rs @@ -1,5 +1,7 @@ -use crate::tracking; -use crate::utils::{resolved_command, tool_exists}; +//! Filters Prisma CLI output by stripping ASCII art and verbose decoration. + +use crate::core::tracking; +use crate::core::utils::{resolved_command, tool_exists}; use anyhow::{Context, Result}; use std::process::Command; diff --git a/src/tsc_cmd.rs b/src/cmds/js/tsc_cmd.rs similarity index 97% rename from src/tsc_cmd.rs rename to src/cmds/js/tsc_cmd.rs index 0758a149..bddd047b 100644 --- a/src/tsc_cmd.rs +++ b/src/cmds/js/tsc_cmd.rs @@ -1,5 +1,7 @@ -use crate::tracking; -use crate::utils::{resolved_command, tool_exists, truncate}; +//! Filters TypeScript compiler errors, grouping them by file and error code. + +use crate::core::tracking; +use crate::core::utils::{resolved_command, tool_exists, truncate}; use anyhow::{Context, Result}; use regex::Regex; use std::collections::HashMap; @@ -37,7 +39,7 @@ pub fn run(args: &[String], verbose: u8) -> Result<()> { let filtered = filter_tsc_output(&raw); let exit_code = output.status.code().unwrap_or(1); - if let Some(hint) = crate::tee::tee_and_hint(&raw, "tsc", exit_code) { + if let Some(hint) = crate::core::tee::tee_and_hint(&raw, "tsc", exit_code) { println!("{}\n{}", filtered, hint); } else { println!("{}", filtered); diff --git a/src/vitest_cmd.rs b/src/cmds/js/vitest_cmd.rs similarity index 98% rename from src/vitest_cmd.rs rename to src/cmds/js/vitest_cmd.rs index 2d8adb31..74d09206 100644 --- a/src/vitest_cmd.rs +++ b/src/cmds/js/vitest_cmd.rs @@ -1,3 +1,5 @@ +//! Filters Vitest test output to show only failures. + use anyhow::{Context, Result}; use regex::Regex; use serde::Deserialize; @@ -6,8 +8,8 @@ use crate::parser::{ emit_degradation_warning, emit_passthrough_warning, extract_json_object, truncate_passthrough, FormatMode, OutputParser, ParseResult, TestFailure, TestResult, TokenFormatter, }; -use crate::tracking; -use crate::utils::{package_manager_exec, strip_ansi}; +use crate::core::tracking; +use crate::core::utils::{package_manager_exec, strip_ansi}; /// Vitest JSON output structures (tool-specific format) #[derive(Debug, Deserialize)] @@ -261,7 +263,7 @@ fn run_vitest(args: &[String], verbose: u8) -> Result<()> { }; let exit_code = output.status.code().unwrap_or(1); - if let Some(hint) = crate::tee::tee_and_hint(&combined, "vitest_run", exit_code) { + if let Some(hint) = crate::core::tee::tee_and_hint(&combined, "vitest_run", exit_code) { println!("{}\n{}", filtered, hint); } else { println!("{}", filtered); diff --git a/src/cmds/mod.rs b/src/cmds/mod.rs new file mode 100644 index 00000000..1eca0b84 --- /dev/null +++ b/src/cmds/mod.rs @@ -0,0 +1,11 @@ +//! Command filter modules organized by language ecosystem. + +pub mod cloud; +pub mod dotnet; +pub mod git; +pub mod go; +pub mod js; +pub mod python; +pub mod ruby; +pub mod rust; +pub mod system; diff --git a/src/cmds/python/README.md b/src/cmds/python/README.md index 5330d66c..5f40ac76 100644 --- a/src/cmds/python/README.md +++ b/src/cmds/python/README.md @@ -1,5 +1,7 @@ # Python Ecosystem +> Part of [`src/cmds/`](../README.md) — see also [docs/TECHNICAL.md](../../../docs/TECHNICAL.md) + ## Specifics - `pytest_cmd.rs` uses a state machine text parser (no JSON available from pytest) diff --git a/src/cmds/python/mod.rs b/src/cmds/python/mod.rs new file mode 100644 index 00000000..7f0a06f2 --- /dev/null +++ b/src/cmds/python/mod.rs @@ -0,0 +1,6 @@ +//! Python ecosystem filters. + +pub mod mypy_cmd; +pub mod pip_cmd; +pub mod pytest_cmd; +pub mod ruff_cmd; diff --git a/src/mypy_cmd.rs b/src/cmds/python/mypy_cmd.rs similarity index 98% rename from src/mypy_cmd.rs rename to src/cmds/python/mypy_cmd.rs index 2cb3ec96..5114452c 100644 --- a/src/mypy_cmd.rs +++ b/src/cmds/python/mypy_cmd.rs @@ -1,5 +1,7 @@ -use crate::tracking; -use crate::utils::{resolved_command, strip_ansi, tool_exists, truncate}; +//! Filters mypy type-checking output, grouping errors by file. + +use crate::core::tracking; +use crate::core::utils::{resolved_command, strip_ansi, tool_exists, truncate}; use anyhow::{Context, Result}; use regex::Regex; use std::collections::HashMap; diff --git a/src/pip_cmd.rs b/src/cmds/python/pip_cmd.rs similarity index 98% rename from src/pip_cmd.rs rename to src/cmds/python/pip_cmd.rs index 1c7dc931..83e3c5b0 100644 --- a/src/pip_cmd.rs +++ b/src/cmds/python/pip_cmd.rs @@ -1,5 +1,7 @@ -use crate::tracking; -use crate::utils::{resolved_command, tool_exists}; +//! Filters pip and uv package manager output. + +use crate::core::tracking; +use crate::core::utils::{resolved_command, tool_exists}; use anyhow::{Context, Result}; use serde::Deserialize; diff --git a/src/pytest_cmd.rs b/src/cmds/python/pytest_cmd.rs similarity index 97% rename from src/pytest_cmd.rs rename to src/cmds/python/pytest_cmd.rs index 0b1b1f2c..412acf9c 100644 --- a/src/pytest_cmd.rs +++ b/src/cmds/python/pytest_cmd.rs @@ -1,5 +1,7 @@ -use crate::tracking; -use crate::utils::{resolved_command, tool_exists, truncate}; +//! Filters pytest output to show only failures and the summary line. + +use crate::core::tracking; +use crate::core::utils::{resolved_command, tool_exists, truncate}; use anyhow::{Context, Result}; #[derive(Debug, PartialEq)] @@ -56,7 +58,7 @@ pub fn run(args: &[String], verbose: u8) -> Result<()> { .status .code() .unwrap_or(if output.status.success() { 0 } else { 1 }); - if let Some(hint) = crate::tee::tee_and_hint(&raw, "pytest", exit_code) { + if let Some(hint) = crate::core::tee::tee_and_hint(&raw, "pytest", exit_code) { println!("{}\n{}", filtered, hint); } else { println!("{}", filtered); diff --git a/src/ruff_cmd.rs b/src/cmds/python/ruff_cmd.rs similarity index 98% rename from src/ruff_cmd.rs rename to src/cmds/python/ruff_cmd.rs index 2cfc2dfa..b0073a8d 100644 --- a/src/ruff_cmd.rs +++ b/src/cmds/python/ruff_cmd.rs @@ -1,6 +1,8 @@ -use crate::config; -use crate::tracking; -use crate::utils::{resolved_command, truncate}; +//! Filters Ruff linter and formatter output. + +use crate::core::config; +use crate::core::tracking; +use crate::core::utils::{resolved_command, truncate}; use anyhow::{Context, Result}; use serde::Deserialize; use std::collections::HashMap; diff --git a/src/cmds/ruby/README.md b/src/cmds/ruby/README.md index 0692458e..08abbc08 100644 --- a/src/cmds/ruby/README.md +++ b/src/cmds/ruby/README.md @@ -1,5 +1,7 @@ # Ruby on Rails +> Part of [`src/cmds/`](../README.md) — see also [docs/TECHNICAL.md](../../../docs/TECHNICAL.md) + ## Specifics - `rake_cmd.rs` filters Minitest output via `rake test` / `rails test`; state machine text parser, failures only (85-90% reduction) diff --git a/src/cmds/ruby/mod.rs b/src/cmds/ruby/mod.rs new file mode 100644 index 00000000..9890d3bd --- /dev/null +++ b/src/cmds/ruby/mod.rs @@ -0,0 +1,5 @@ +//! Ruby ecosystem filters. + +pub mod rake_cmd; +pub mod rspec_cmd; +pub mod rubocop_cmd; diff --git a/src/rake_cmd.rs b/src/cmds/ruby/rake_cmd.rs similarity index 98% rename from src/rake_cmd.rs rename to src/cmds/ruby/rake_cmd.rs index e3fba68f..c0f21f61 100644 --- a/src/rake_cmd.rs +++ b/src/cmds/ruby/rake_cmd.rs @@ -4,8 +4,8 @@ //! `rails test`, filtering down to failures/errors and the summary line. //! Uses `ruby_exec("rake")` to auto-detect `bundle exec`. -use crate::tracking; -use crate::utils::{exit_code_from_output, ruby_exec, strip_ansi}; +use crate::core::tracking; +use crate::core::utils::{exit_code_from_output, ruby_exec, strip_ansi}; use anyhow::{Context, Result}; /// Decide whether to use `rake test` or `rails test` based on args. @@ -74,7 +74,7 @@ pub fn run(args: &[String], verbose: u8) -> Result<()> { let filtered = filter_minitest_output(&raw); let exit_code = exit_code_from_output(&output, "rake"); - if let Some(hint) = crate::tee::tee_and_hint(&raw, "rake", exit_code) { + if let Some(hint) = crate::core::tee::tee_and_hint(&raw, "rake", exit_code) { println!("{}\n{}", filtered, hint); } else { println!("{}", filtered); @@ -236,7 +236,7 @@ fn build_minitest_summary(summary: &str, failures: &[String]) -> String { for line in lines.iter().skip(1).take(4) { let trimmed = line.trim(); if !trimmed.is_empty() { - result.push_str(&format!(" {}\n", crate::utils::truncate(trimmed, 120))); + result.push_str(&format!(" {}\n", crate::core::utils::truncate(trimmed, 120))); } } if i < failures.len().min(10) - 1 { @@ -281,7 +281,7 @@ fn parse_minitest_summary(summary: &str) -> (usize, usize, usize, usize, usize) #[cfg(test)] mod tests { use super::*; - use crate::utils::count_tokens; + use crate::core::utils::count_tokens; #[test] fn test_filter_minitest_all_pass() { diff --git a/src/rspec_cmd.rs b/src/cmds/ruby/rspec_cmd.rs similarity index 99% rename from src/rspec_cmd.rs rename to src/cmds/ruby/rspec_cmd.rs index 3d8bf2c4..7b3d94db 100644 --- a/src/rspec_cmd.rs +++ b/src/cmds/ruby/rspec_cmd.rs @@ -5,8 +5,8 @@ //! (e.g., user specified `--format documentation`) or when injected JSON output //! fails to parse. -use crate::tracking; -use crate::utils::{exit_code_from_output, fallback_tail, ruby_exec, truncate}; +use crate::core::tracking; +use crate::core::utils::{exit_code_from_output, fallback_tail, ruby_exec, truncate}; use anyhow::{Context, Result}; use lazy_static::lazy_static; use regex::Regex; @@ -107,7 +107,7 @@ pub fn run(args: &[String], verbose: u8) -> Result<()> { filter_rspec_output(&stdout) }; - if let Some(hint) = crate::tee::tee_and_hint(&raw, "rspec", exit_code) { + if let Some(hint) = crate::core::tee::tee_and_hint(&raw, "rspec", exit_code) { println!("{}\n{}", filtered, hint); } else { println!("{}", filtered); @@ -457,7 +457,7 @@ fn compact_failure_block(block: &str) -> String { #[cfg(test)] mod tests { use super::*; - use crate::utils::count_tokens; + use crate::core::utils::count_tokens; fn all_pass_json() -> &'static str { r#"{ diff --git a/src/rubocop_cmd.rs b/src/cmds/ruby/rubocop_cmd.rs similarity index 98% rename from src/rubocop_cmd.rs rename to src/cmds/ruby/rubocop_cmd.rs index db2d0ac4..14342d8f 100644 --- a/src/rubocop_cmd.rs +++ b/src/cmds/ruby/rubocop_cmd.rs @@ -5,8 +5,8 @@ //! when the user specifies a custom format, or when injected JSON output fails //! to parse. -use crate::tracking; -use crate::utils::{exit_code_from_output, ruby_exec}; +use crate::core::tracking; +use crate::core::utils::{exit_code_from_output, ruby_exec}; use anyhow::{Context, Result}; use serde::Deserialize; @@ -93,7 +93,7 @@ pub fn run(args: &[String], verbose: u8) -> Result<()> { filter_rubocop_json(&stdout) }; - if let Some(hint) = crate::tee::tee_and_hint(&raw, "rubocop", exit_code) { + if let Some(hint) = crate::core::tee::tee_and_hint(&raw, "rubocop", exit_code) { println!("{}\n{}", filtered, hint); } else { println!("{}", filtered); @@ -139,7 +139,7 @@ fn filter_rubocop_json(output: &str) -> String { Ok(r) => r, Err(e) => { eprintln!("[rtk] rubocop: JSON parse failed ({})", e); - return crate::utils::fallback_tail(output, "rubocop (JSON parse error)", 5); + return crate::core::utils::fallback_tail(output, "rubocop (JSON parse error)", 5); } }; @@ -291,7 +291,7 @@ fn filter_rubocop_text(output: &str) -> String { } } // Last resort: last 5 lines - crate::utils::fallback_tail(output, "rubocop", 5) + crate::core::utils::fallback_tail(output, "rubocop", 5) } /// Extract leading number from a string like "15 files inspected". @@ -353,7 +353,7 @@ fn compact_ruby_path(path: &str) -> String { #[cfg(test)] mod tests { use super::*; - use crate::utils::count_tokens; + use crate::core::utils::count_tokens; fn no_offenses_json() -> &'static str { r#"{ diff --git a/src/cmds/rust/README.md b/src/cmds/rust/README.md index 42b6ac9d..f7f9204c 100644 --- a/src/cmds/rust/README.md +++ b/src/cmds/rust/README.md @@ -1,5 +1,7 @@ # Rust Ecosystem +> Part of [`src/cmds/`](../README.md) — see also [docs/TECHNICAL.md](../../../docs/TECHNICAL.md) + ## Specifics - `cargo_cmd.rs` uses `restore_double_dash()` fix: Clap strips `--` but cargo needs it for test flags (e.g., `cargo test -- --nocapture`) diff --git a/src/cargo_cmd.rs b/src/cmds/rust/cargo_cmd.rs similarity index 99% rename from src/cargo_cmd.rs rename to src/cmds/rust/cargo_cmd.rs index eabf8a37..90416a19 100644 --- a/src/cargo_cmd.rs +++ b/src/cmds/rust/cargo_cmd.rs @@ -1,5 +1,7 @@ -use crate::tracking; -use crate::utils::{resolved_command, truncate}; +//! Filters cargo output — build errors, test results, clippy warnings. + +use crate::core::tracking; +use crate::core::utils::{resolved_command, truncate}; use anyhow::{Context, Result}; use std::collections::HashMap; use std::ffi::OsString; @@ -97,7 +99,7 @@ where .unwrap_or(if output.status.success() { 0 } else { 1 }); let filtered = filter_fn(&raw); - if let Some(hint) = crate::tee::tee_and_hint(&raw, &format!("cargo_{}", subcommand), exit_code) + if let Some(hint) = crate::core::tee::tee_and_hint(&raw, &format!("cargo_{}", subcommand), exit_code) { println!("{}\n{}", filtered, hint); } else { diff --git a/src/cmds/rust/mod.rs b/src/cmds/rust/mod.rs new file mode 100644 index 00000000..c58ffbad --- /dev/null +++ b/src/cmds/rust/mod.rs @@ -0,0 +1,4 @@ +//! Rust ecosystem filters. + +pub mod cargo_cmd; +pub mod runner; diff --git a/src/runner.rs b/src/cmds/rust/runner.rs similarity index 96% rename from src/runner.rs rename to src/cmds/rust/runner.rs index 1a2eceed..8c3f3527 100644 --- a/src/runner.rs +++ b/src/cmds/rust/runner.rs @@ -1,4 +1,6 @@ -use crate::tracking; +//! Runs arbitrary commands and captures only stderr or test failures. + +use crate::core::tracking; use anyhow::{Context, Result}; use regex::Regex; use std::process::{Command, Stdio}; @@ -53,7 +55,7 @@ pub fn run_err(command: &str, verbose: u8) -> Result<()> { .status .code() .unwrap_or(if output.status.success() { 0 } else { 1 }); - if let Some(hint) = crate::tee::tee_and_hint(&raw, "err", exit_code) { + if let Some(hint) = crate::core::tee::tee_and_hint(&raw, "err", exit_code) { println!("{}\n{}", rtk, hint); } else { println!("{}", rtk); @@ -94,7 +96,7 @@ pub fn run_test(command: &str, verbose: u8) -> Result<()> { .code() .unwrap_or(if output.status.success() { 0 } else { 1 }); let summary = extract_test_summary(&raw, command); - if let Some(hint) = crate::tee::tee_and_hint(&raw, "test", exit_code) { + if let Some(hint) = crate::core::tee::tee_and_hint(&raw, "test", exit_code) { println!("{}\n{}", summary, hint); } else { println!("{}", summary); diff --git a/src/cmds/system/README.md b/src/cmds/system/README.md index fd503bdd..bffdff17 100644 --- a/src/cmds/system/README.md +++ b/src/cmds/system/README.md @@ -1,5 +1,7 @@ # System and Generic Utilities +> Part of [`src/cmds/`](../README.md) — see also [docs/TECHNICAL.md](../../../docs/TECHNICAL.md) + ## Specifics - `read.rs` uses `core/filter` for language-aware code stripping (FilterLevel: none/minimal/aggressive) diff --git a/src/deps.rs b/src/cmds/system/deps.rs similarity index 98% rename from src/deps.rs rename to src/cmds/system/deps.rs index 27902984..0342b2be 100644 --- a/src/deps.rs +++ b/src/cmds/system/deps.rs @@ -1,4 +1,6 @@ -use crate::tracking; +//! Summarizes project dependencies from lock files and manifests. + +use crate::core::tracking; use anyhow::Result; use regex::Regex; use std::fs; diff --git a/src/env_cmd.rs b/src/cmds/system/env_cmd.rs similarity index 98% rename from src/env_cmd.rs rename to src/cmds/system/env_cmd.rs index d4b9b6a3..3b830fe4 100644 --- a/src/env_cmd.rs +++ b/src/cmds/system/env_cmd.rs @@ -1,4 +1,6 @@ -use crate::tracking; +//! Filters environment variables, hiding secrets and noise. + +use crate::core::tracking; use anyhow::Result; use std::collections::HashSet; use std::env; diff --git a/src/find_cmd.rs b/src/cmds/system/find_cmd.rs similarity index 99% rename from src/find_cmd.rs rename to src/cmds/system/find_cmd.rs index df1e41b2..942843fb 100644 --- a/src/find_cmd.rs +++ b/src/cmds/system/find_cmd.rs @@ -1,4 +1,6 @@ -use crate::tracking; +//! Filters find results by grouping files by directory. + +use crate::core::tracking; use anyhow::{Context, Result}; use ignore::WalkBuilder; use std::collections::HashMap; diff --git a/src/format_cmd.rs b/src/cmds/system/format_cmd.rs similarity index 98% rename from src/format_cmd.rs rename to src/cmds/system/format_cmd.rs index 23c01a2b..b9f018f6 100644 --- a/src/format_cmd.rs +++ b/src/cmds/system/format_cmd.rs @@ -1,7 +1,9 @@ +//! Runs code formatters (Prettier, Ruff) and shows only files that changed. + use crate::prettier_cmd; use crate::ruff_cmd; -use crate::tracking; -use crate::utils::{package_manager_exec, resolved_command}; +use crate::core::tracking; +use crate::core::utils::{package_manager_exec, resolved_command}; use anyhow::{Context, Result}; use std::path::Path; diff --git a/src/grep_cmd.rs b/src/cmds/system/grep_cmd.rs similarity index 98% rename from src/grep_cmd.rs rename to src/cmds/system/grep_cmd.rs index c1819dde..56f8cb6a 100644 --- a/src/grep_cmd.rs +++ b/src/cmds/system/grep_cmd.rs @@ -1,6 +1,8 @@ -use crate::config; -use crate::tracking; -use crate::utils::resolved_command; +//! Filters grep output by grouping matches by file. + +use crate::core::config; +use crate::core::tracking; +use crate::core::utils::resolved_command; use anyhow::{Context, Result}; use regex::Regex; use std::collections::HashMap; diff --git a/src/json_cmd.rs b/src/cmds/system/json_cmd.rs similarity index 99% rename from src/json_cmd.rs rename to src/cmds/system/json_cmd.rs index 685c8f62..4e887417 100644 --- a/src/json_cmd.rs +++ b/src/cmds/system/json_cmd.rs @@ -1,4 +1,6 @@ -use crate::tracking; +//! Inspects JSON structure without showing values, saving tokens on large payloads. + +use crate::core::tracking; use anyhow::{bail, Context, Result}; use serde_json::Value; use std::fs; diff --git a/src/local_llm.rs b/src/cmds/system/local_llm.rs similarity index 98% rename from src/local_llm.rs rename to src/cmds/system/local_llm.rs index ec0dcf53..20ac7c18 100644 --- a/src/local_llm.rs +++ b/src/cmds/system/local_llm.rs @@ -1,9 +1,11 @@ +//! Summarizes source files using heuristic analysis — no external model needed. + use anyhow::{Context, Result}; use regex::Regex; use std::fs; use std::path::Path; -use crate::filter::Language; +use crate::core::filter::Language; /// Heuristic-based code summarizer - no external model needed pub fn run(file: &Path, _model: &str, _force_download: bool, verbose: u8) -> Result<()> { diff --git a/src/log_cmd.rs b/src/cmds/system/log_cmd.rs similarity index 98% rename from src/log_cmd.rs rename to src/cmds/system/log_cmd.rs index 0deadf90..fd9942d0 100644 --- a/src/log_cmd.rs +++ b/src/cmds/system/log_cmd.rs @@ -1,4 +1,6 @@ -use crate::tracking; +//! Deduplicates repeated log lines and shows counts instead. + +use crate::core::tracking; use anyhow::Result; use lazy_static::lazy_static; use regex::Regex; diff --git a/src/ls.rs b/src/cmds/system/ls.rs similarity index 98% rename from src/ls.rs rename to src/cmds/system/ls.rs index d121123a..3426e7fb 100644 --- a/src/ls.rs +++ b/src/cmds/system/ls.rs @@ -1,5 +1,7 @@ -use crate::tracking; -use crate::utils::resolved_command; +//! Filters directory listings into a compact tree format. + +use crate::core::tracking; +use crate::core::utils::resolved_command; use anyhow::{Context, Result}; /// Noise directories commonly excluded from LLM context diff --git a/src/cmds/system/mod.rs b/src/cmds/system/mod.rs new file mode 100644 index 00000000..a7686922 --- /dev/null +++ b/src/cmds/system/mod.rs @@ -0,0 +1,15 @@ +//! General-purpose system command filters. + +pub mod deps; +pub mod env_cmd; +pub mod find_cmd; +pub mod format_cmd; +pub mod grep_cmd; +pub mod json_cmd; +pub mod local_llm; +pub mod log_cmd; +pub mod ls; +pub mod read; +pub mod summary; +pub mod tree; +pub mod wc_cmd; diff --git a/src/read.rs b/src/cmds/system/read.rs similarity index 97% rename from src/read.rs rename to src/cmds/system/read.rs index 262ef452..23c45afb 100644 --- a/src/read.rs +++ b/src/cmds/system/read.rs @@ -1,5 +1,7 @@ -use crate::filter::{self, FilterLevel, Language}; -use crate::tracking; +//! Reads source files with optional language-aware filtering to strip boilerplate. + +use crate::core::filter::{self, FilterLevel, Language}; +use crate::core::tracking; use anyhow::{Context, Result}; use std::fs; use std::path::Path; diff --git a/src/summary.rs b/src/cmds/system/summary.rs similarity index 98% rename from src/summary.rs rename to src/cmds/system/summary.rs index a295b73d..be44f883 100644 --- a/src/summary.rs +++ b/src/cmds/system/summary.rs @@ -1,5 +1,7 @@ -use crate::tracking; -use crate::utils::truncate; +//! Runs a command and produces a heuristic summary of its output. + +use crate::core::tracking; +use crate::core::utils::truncate; use anyhow::{Context, Result}; use regex::Regex; use std::process::{Command, Stdio}; diff --git a/src/tree.rs b/src/cmds/system/tree.rs similarity index 98% rename from src/tree.rs rename to src/cmds/system/tree.rs index 4727a740..57706a6a 100644 --- a/src/tree.rs +++ b/src/cmds/system/tree.rs @@ -6,8 +6,8 @@ //! Token optimization: automatically excludes noise directories via -I pattern //! unless -a flag is present (respecting user intent). -use crate::tracking; -use crate::utils::{resolved_command, tool_exists}; +use crate::core::tracking; +use crate::core::utils::{resolved_command, tool_exists}; use anyhow::{Context, Result}; /// Noise directories commonly excluded from LLM context diff --git a/src/wc_cmd.rs b/src/cmds/system/wc_cmd.rs similarity index 99% rename from src/wc_cmd.rs rename to src/cmds/system/wc_cmd.rs index 7cd01998..14e4f69f 100644 --- a/src/wc_cmd.rs +++ b/src/cmds/system/wc_cmd.rs @@ -6,8 +6,8 @@ /// - `wc -w file.py` → `96` /// - `wc -c file.py` → `978` /// - `wc -l *.py` → table with common path prefix stripped -use crate::tracking; -use crate::utils::resolved_command; +use crate::core::tracking; +use crate::core::utils::resolved_command; use anyhow::{Context, Result}; pub fn run(args: &[String], verbose: u8) -> Result<()> { diff --git a/src/core/README.md b/src/core/README.md index 2f0127f7..2f0a4684 100644 --- a/src/core/README.md +++ b/src/core/README.md @@ -1,5 +1,7 @@ # Core Infrastructure +> See also [docs/TECHNICAL.md](../../docs/TECHNICAL.md) for the full architecture overview + ## Scope Domain-agnostic building blocks with **no knowledge of any specific command, hook, or agent**. If a module references "git", "cargo", "claude", or any external tool by name, it does not belong here. Core is a leaf in the dependency graph — it is consumed by all other components but imports from none of them. @@ -9,19 +11,7 @@ Owns: configuration loading, token tracking persistence, TOML filter engine, tee Does **not** own: command-specific filtering logic (that's `cmds/`), hook lifecycle management (that's `src/hooks/`), or analytics dashboards (that's `analytics/`). ## Purpose -Core infrastructure shared by all RTK command modules. These are the foundational building blocks that every filter, tracker, and command handler depends on. This module group has no inward dependencies -- it is a leaf in the dependency graph, ensuring clean layering across the codebase. - -## Files -| File | Responsibility | -|------|---------------| -| tracking.rs | SQLite-based persistent token metrics (1429 lines); records input/output token counts per command with project path; 90-day retention cleanup; token estimation via `ceil(chars / 4.0)` heuristic; `TimedExecution` API for timer-based tracking; imported by 46 files (highest dependency count); DB location: `RTK_DB_PATH` env > `config.toml` > `~/.local/share/rtk/tracking.db` | -| utils.rs | Shared utilities used across 41 files: `strip_ansi`, `truncate`, `execute_command`, `resolved_command`, `package_manager_exec`; package manager auto-detection (pnpm/yarn/npm/npx); consistent error handling and output formatting | -| config.rs | Configuration system reading `~/.config/rtk/config.toml`; sections: `[tracking]` (DB path, retention), `[display]` (colors, emoji, max_width), `[tee]` (recovery), `[telemetry]`, `[hooks]` (exclude_commands), `[limits]` (grep/status thresholds); on-demand loading (no startup I/O) | -| filter.rs | Language-aware code filtering engine with `FilterLevel` enum (`none`, `minimal`, `aggressive`); strips comments, whitespace, and function bodies based on level; supports Rust, Python, JS/TS, Java, Go, C/C++ via file extension detection with fallback heuristics | -| toml_filter.rs | TOML DSL filter engine (1686 lines); `TomlFilterRegistry` singleton via `lazy_static!`; three-tier lookup: project-local (trust-gated), user-global, built-in; 8-stage filter pipeline (see below); built-in filters compiled from `src/filters/*.toml` via `build.rs` | -| tee.rs | Raw output recovery on command failure (400 lines); saves unfiltered output to `~/.local/share/rtk/tee/{epoch}_{slug}.log`; prints one-line hint for LLM re-read; 20-file rotation, 1MB cap, min 500 chars; configurable via `[tee]` config section or `RTK_TEE`/`RTK_TEE_DIR` env vars; tee errors never affect command output or exit code | -| display_helpers.rs | Token display formatting helpers for consistent human-readable output of savings percentages, token counts, and comparison tables | -| telemetry.rs | Fire-and-forget usage telemetry (248 lines); non-blocking background thread; once per 23 hours via marker file; sends device hash (SHA-256 of hostname:username), version, OS, top commands, savings stats; 2-second timeout; disabled via `RTK_TELEMETRY_DISABLED=1` or `[telemetry] enabled = false` | +Core infrastructure shared by all RTK command modules. These are the foundational building blocks that every filter, tracker, and command handler depends on. This module group has no inward dependencies — it is a leaf in the dependency graph, ensuring clean layering across the codebase. ## TOML Filter Pipeline @@ -102,6 +92,21 @@ status_max_untracked = 10 passthrough_max_chars = 2000 ``` +## Shared Utilities (utils.rs) + +Key functions available to all command modules (41 modules depend on `core::utils`): + +| Function | Purpose | +|----------|---------| +| `truncate(s, max)` | Truncate string with `...` suffix | +| `strip_ansi(text)` | Remove ANSI escape/color codes | +| `resolved_command(name)` | Find command in PATH, returns `Command` | +| `tool_exists(name)` | Check if a CLI tool is available | +| `detect_package_manager()` | Detect pnpm/yarn/npm from lockfiles | +| `package_manager_exec(tool)` | Build `Command` using detected package manager | +| `ruby_exec(tool)` | Auto-detect `bundle exec` when `Gemfile` exists | +| `count_tokens(text)` | Estimate tokens: `ceil(chars / 4.0)` | + ## Consumer Contracts Core provides infrastructure that `cmds/` and other components consume. These contracts define expected usage. diff --git a/src/config.rs b/src/core/config.rs similarity index 98% rename from src/config.rs rename to src/core/config.rs index 7ffea86d..99e28c21 100644 --- a/src/config.rs +++ b/src/core/config.rs @@ -1,3 +1,5 @@ +//! Reads user settings from config.toml. + use anyhow::Result; use serde::{Deserialize, Serialize}; use std::path::PathBuf; @@ -11,7 +13,7 @@ pub struct Config { #[serde(default)] pub filters: FilterConfig, #[serde(default)] - pub tee: crate::tee::TeeConfig, + pub tee: crate::core::tee::TeeConfig, #[serde(default)] pub telemetry: TelemetryConfig, #[serde(default)] diff --git a/src/display_helpers.rs b/src/core/display_helpers.rs similarity index 98% rename from src/display_helpers.rs rename to src/core/display_helpers.rs index 60354c7c..affc331b 100644 --- a/src/display_helpers.rs +++ b/src/core/display_helpers.rs @@ -1,10 +1,10 @@ -//! Generic table display helpers for period-based statistics +//! Formats token counts and savings tables for terminal display. //! //! Eliminates duplication in gain.rs and cc_economics.rs by providing //! a unified trait-based system for displaying daily/weekly/monthly data. -use crate::tracking::{DayStats, MonthStats, WeekStats}; -use crate::utils::format_tokens; +use crate::core::tracking::{DayStats, MonthStats, WeekStats}; +use crate::core::utils::format_tokens; /// Format duration in milliseconds to human-readable string pub fn format_duration(ms: u64) -> String { diff --git a/src/filter.rs b/src/core/filter.rs similarity index 99% rename from src/filter.rs rename to src/core/filter.rs index d6d9d19b..475074fd 100644 --- a/src/filter.rs +++ b/src/core/filter.rs @@ -1,3 +1,5 @@ +//! Strips comments and boilerplate from source code to save tokens. + use lazy_static::lazy_static; use regex::Regex; use std::str::FromStr; diff --git a/src/core/mod.rs b/src/core/mod.rs new file mode 100644 index 00000000..0a490aad --- /dev/null +++ b/src/core/mod.rs @@ -0,0 +1,10 @@ +//! Building blocks shared across all RTK modules. + +pub mod config; +pub mod display_helpers; +pub mod filter; +pub mod tee; +pub mod telemetry; +pub mod toml_filter; +pub mod tracking; +pub mod utils; diff --git a/src/tee.rs b/src/core/tee.rs similarity index 99% rename from src/tee.rs rename to src/core/tee.rs index 1dbbe4e8..a4cf3ccc 100644 --- a/src/tee.rs +++ b/src/core/tee.rs @@ -1,4 +1,6 @@ -use crate::config::Config; +//! Raw output recovery -- saves unfiltered output to disk on command failure. + +use crate::core::config::Config; use std::path::PathBuf; /// Minimum output size to tee (smaller outputs don't need recovery) diff --git a/src/telemetry.rs b/src/core/telemetry.rs similarity index 98% rename from src/telemetry.rs rename to src/core/telemetry.rs index 36b1e724..e53e05fd 100644 --- a/src/telemetry.rs +++ b/src/core/telemetry.rs @@ -1,5 +1,7 @@ -use crate::config; -use crate::tracking; +//! Optional usage ping so we know which commands people run most. + +use crate::core::config; +use crate::core::tracking; use sha2::{Digest, Sha256}; use std::path::PathBuf; diff --git a/src/toml_filter.rs b/src/core/toml_filter.rs similarity index 98% rename from src/toml_filter.rs rename to src/core/toml_filter.rs index 0f571626..b2047e61 100644 --- a/src/toml_filter.rs +++ b/src/core/toml_filter.rs @@ -1,4 +1,4 @@ -/// TOML-based filter DSL for RTK. +//! Applies TOML-defined filter rules to command output. /// /// Provides a declarative pipeline of 8 stages that can be configured /// via TOML files. Lookup priority (first match wins): @@ -184,11 +184,11 @@ impl TomlFilterRegistry { // Priority 1: project-local .rtk/filters.toml (trust-gated) let project_filter_path = std::path::Path::new(".rtk/filters.toml"); if project_filter_path.exists() { - let trust_status = crate::trust::check_trust(project_filter_path) - .unwrap_or(crate::trust::TrustStatus::Untrusted); + let trust_status = crate::hooks::trust::check_trust(project_filter_path) + .unwrap_or(crate::hooks::trust::TrustStatus::Untrusted); match trust_status { - crate::trust::TrustStatus::Trusted | crate::trust::TrustStatus::EnvOverride => { + crate::hooks::trust::TrustStatus::Trusted | crate::hooks::trust::TrustStatus::EnvOverride => { if let Ok(content) = std::fs::read_to_string(project_filter_path) { match Self::parse_and_compile(&content, "project") { Ok(f) => filters.extend(f), @@ -196,11 +196,11 @@ impl TomlFilterRegistry { } } } - crate::trust::TrustStatus::Untrusted => { + crate::hooks::trust::TrustStatus::Untrusted => { eprintln!("[rtk] WARNING: untrusted project filters (.rtk/filters.toml)"); eprintln!("[rtk] Filters NOT applied. Run `rtk trust` to review and enable."); } - crate::trust::TrustStatus::ContentChanged { .. } => { + crate::hooks::trust::TrustStatus::ContentChanged { .. } => { eprintln!("[rtk] WARNING: .rtk/filters.toml changed since trusted."); eprintln!("[rtk] Filters NOT applied. Run `rtk trust` to re-review."); } @@ -431,7 +431,7 @@ pub fn apply_filter(filter: &CompiledFilter, stdout: &str) -> String { if filter.strip_ansi { lines = lines .into_iter() - .map(|l| crate::utils::strip_ansi(&l)) + .map(|l| crate::core::utils::strip_ansi(&l)) .collect(); } @@ -478,7 +478,7 @@ pub fn apply_filter(filter: &CompiledFilter, stdout: &str) -> String { if let Some(max_chars) = filter.truncate_lines_at { lines = lines .into_iter() - .map(|l| crate::utils::truncate(&l, max_chars)) + .map(|l| crate::core::utils::truncate(&l, max_chars)) .collect(); } @@ -551,9 +551,9 @@ pub fn run_filter_tests(filter_name_opt: Option<&str>) -> VerifyResults { let project_path = std::path::Path::new(".rtk/filters.toml"); if project_path.exists() { let trust_status = - crate::trust::check_trust(project_path).unwrap_or(crate::trust::TrustStatus::Untrusted); + crate::hooks::trust::check_trust(project_path).unwrap_or(crate::hooks::trust::TrustStatus::Untrusted); match trust_status { - crate::trust::TrustStatus::Trusted | crate::trust::TrustStatus::EnvOverride => { + crate::hooks::trust::TrustStatus::Trusted | crate::hooks::trust::TrustStatus::EnvOverride => { if let Ok(content) = std::fs::read_to_string(project_path) { collect_test_outcomes( &content, diff --git a/src/tracking.rs b/src/core/tracking.rs similarity index 99% rename from src/tracking.rs rename to src/core/tracking.rs index dd73788a..1cba65fd 100644 --- a/src/tracking.rs +++ b/src/core/tracking.rs @@ -966,7 +966,7 @@ fn get_db_path() -> Result { } // Priority 2: Configuration file - if let Ok(config) = crate::config::Config::load() { + if let Ok(config) = crate::core::config::Config::load() { if let Some(db_path) = config.tracking.database_path { return Ok(db_path); } diff --git a/src/utils.rs b/src/core/utils.rs similarity index 100% rename from src/utils.rs rename to src/core/utils.rs diff --git a/src/discover/README.md b/src/discover/README.md new file mode 100644 index 00000000..116d059b --- /dev/null +++ b/src/discover/README.md @@ -0,0 +1,32 @@ +# Discover — Claude Code History Analysis + +> See also [docs/TECHNICAL.md](../../docs/TECHNICAL.md) for the full architecture overview + +## Purpose + +Scans Claude Code JSONL session files to identify commands that could benefit from RTK filtering. Powers the `rtk discover` command, which reports missed savings opportunities and adoption metrics. + +Also provides the **command rewrite registry** — the single source of truth for all 70+ patterns used by every LLM agent hook to decide which commands to rewrite. + +## Key Types + +- **`Classification`** — Result of `classify_command()`: `Supported { rtk_equivalent, category, savings_pct, status }`, `Unsupported { base_command }`, or `Ignored` +- **`RtkStatus`** — `Existing` (dedicated handler), `Passthrough` (external_subcommand), `NotSupported` +- **`SessionProvider`** trait — abstraction for session file discovery (currently only `ClaudeProvider`) +- **`ExtractedCommand`** — command string + output length + error flag extracted from JSONL + +## Dependencies + +- **Uses**: `walkdir` (session file discovery), `lazy_static`/`regex` (pattern matching), `serde_json` (JSONL parsing) +- **Used by**: `src/hooks/rewrite_cmd.rs` (imports `registry::classify_command` for `rtk rewrite`), `src/learn/` (imports `provider::ClaudeProvider` for session extraction), `src/main.rs` (routes `rtk discover` command) + +## Registry Architecture + +`registry.rs` (2270 lines) is the largest file in the project. It contains: + +1. **Pattern matching** — Compiled regexes in `lazy_static!` matching command prefixes (e.g., `^git\s+(status|log|diff|...)`) +2. **Compound splitting** — `split_command_chain()` handles `&&`, `||`, `;`, `|`, `&` operators with shell quoting awareness +3. **RTK_DISABLED detection** — `has_rtk_disabled_prefix()` / `strip_disabled_prefix()` for per-command override +4. **Category averages** — `category_avg_tokens()` estimates output tokens when real data unavailable + +The registry is used by both `rtk discover` (analysis) and `rtk rewrite` (live rewriting). Same patterns, different consumers. diff --git a/src/discover/mod.rs b/src/discover/mod.rs index ba868a03..4be67a50 100644 --- a/src/discover/mod.rs +++ b/src/discover/mod.rs @@ -1,3 +1,5 @@ +//! Scans AI coding sessions to find commands that could benefit from RTK filtering. + pub mod provider; pub mod registry; mod report; diff --git a/src/discover/provider.rs b/src/discover/provider.rs index b4105a9d..286c3891 100644 --- a/src/discover/provider.rs +++ b/src/discover/provider.rs @@ -1,3 +1,5 @@ +//! Reads Claude Code session logs from disk and streams their command history. + use anyhow::{Context, Result}; use std::collections::HashMap; use std::fs; diff --git a/src/discover/registry.rs b/src/discover/registry.rs index d04a112a..863b1137 100644 --- a/src/discover/registry.rs +++ b/src/discover/registry.rs @@ -1,3 +1,5 @@ +//! Matches shell commands against known RTK rewrite rules to decide how to handle them. + use lazy_static::lazy_static; use regex::{Regex, RegexSet}; diff --git a/src/discover/report.rs b/src/discover/report.rs index 5b1fe801..1fffa327 100644 --- a/src/discover/report.rs +++ b/src/discover/report.rs @@ -1,3 +1,5 @@ +//! Data types for reporting which commands RTK can and cannot optimize. + use serde::Serialize; /// RTK support status for a command. diff --git a/src/discover/rules.rs b/src/discover/rules.rs index 44f19d60..d72429b0 100644 --- a/src/discover/rules.rs +++ b/src/discover/rules.rs @@ -1,3 +1,5 @@ +//! The master list of shell commands RTK knows how to rewrite. + use super::report::RtkStatus; /// A rule mapping a shell command pattern to its RTK equivalent. diff --git a/src/filters/README.md b/src/filters/README.md index b7d7487c..e6bf7453 100644 --- a/src/filters/README.md +++ b/src/filters/README.md @@ -1,8 +1,22 @@ # Built-in Filters +> See also [docs/TECHNICAL.md](../../docs/TECHNICAL.md) for the full architecture overview + Each `.toml` file in this directory defines one filter and its inline tests. Files are concatenated alphabetically by `build.rs` into a single TOML blob embedded in the binary. +## When to Use a TOML Filter + +TOML filters strip noise lines — they don't reformat output. The filtered result must still look like real command output (see [Design Philosophy](../../CONTRIBUTING.md#design-philosophy)). For the full TOML-vs-Rust decision criteria, see [CONTRIBUTING.md](../../CONTRIBUTING.md#toml-vs-rust-which-one). + +TOML works well for commands with **predictable, line-by-line text output** where regex filtering achieves 60%+ savings: +- Install/update logs (brew, composer, poetry) — strip `Using ...` / `Already installed` lines +- System monitoring (df, ps, systemctl) — keep essential rows, drop headers/decorations +- Simple linters (shellcheck, yamllint, hadolint) — strip context, keep findings +- Infra tools (terraform plan, helm, rsync) — strip progress, keep summary + +For the full contribution checklist (including `discover/rules.rs` registration), see [CONTRIBUTING.md](../../CONTRIBUTING.md#complete-contribution-checklist). + ## Adding a filter 1. Copy any existing `.toml` file and rename it (e.g. `my-tool.toml`) diff --git a/src/hooks/README.md b/src/hooks/README.md index 0e46ca94..65e05f03 100644 --- a/src/hooks/README.md +++ b/src/hooks/README.md @@ -1,5 +1,7 @@ # Hook System +> See also [docs/TECHNICAL.md](../../docs/TECHNICAL.md) for the full architecture overview | [hooks/](../../hooks/README.md) for deployed hook artifacts + ## Scope The **lifecycle management** layer for LLM agent hooks: install, uninstall, verify integrity, audit usage, and manage trust. This component creates and maintains the hook artifacts that live in `hooks/` (root), but does **not** execute rewrite logic itself — that lives in `discover/registry`. @@ -15,18 +17,6 @@ Boundary notes: ## Purpose LLM agent integration layer that installs, validates, and executes command-rewriting hooks for AI coding assistants. Hooks intercept raw CLI commands (e.g., `git status`) and rewrite them to RTK equivalents (e.g., `rtk git status`) so that LLM agents automatically benefit from token savings without explicit user configuration. -## Files -| File | Responsibility | -|------|---------------| -| init.rs | `rtk init` command (2998 lines) -- orchestrates all installation/uninstallation flows for 4 agents (Claude, Cursor, Windsurf, Cline) + 3 special modes (Gemini, Codex, OpenCode); supports 6 installation modes (default, hook-only, claude-md, windsurf, cline, codex); handles settings.json patching, RTK.md writing, CLAUDE.md injection, and OpenCode/Cursor side-installs | -| hook_cmd.rs | Hook processors for Gemini CLI (`run_gemini()`) and GitHub Copilot (`run_copilot()`); reads JSON from stdin, auto-detects agent format (VS Code vs Copilot CLI), calls `rewrite_command()` in-process, returns agent-specific JSON response | -| hook_check.rs | Runtime hook version detection; parses `# rtk-hook-version: N` from hook script header; warns if outdated or missing; rate-limited to once per 24 hours via marker file at `~/.local/share/rtk/.hook_warn_last` | -| hook_audit_cmd.rs | `rtk hook-audit` command; analyzes hook audit log (`~/.local/share/rtk/hook-audit.log`) enabled via `RTK_HOOK_AUDIT=1`; shows rewrite success rates, skip reasons, and top commands | -| rewrite_cmd.rs | `rtk rewrite` command -- thin wrapper that loads config exclusions and delegates to `discover/registry::rewrite_command()`; used by all shell-based hooks as a subprocess | -| verify_cmd.rs | `rtk verify` command -- runs inline tests from TOML filter files; integrity verification (`integrity::run_verify()`) is routed via `main.rs`, not this module | -| trust.rs | `rtk trust` / `rtk untrust` commands -- manages a trust store for project-local TOML filters in `.rtk/filters/`; prevents untrusted filters from executing | -| integrity.rs | SHA-256 hook integrity system (538 lines); computes and stores hashes at install time; verifies at runtime; 5-state model: Verified, Tampered, NoBaseline, NotInstalled, OrphanedHash | - ## Installation Modes `rtk init` supports 6 distinct installation flows: diff --git a/src/hook_audit_cmd.rs b/src/hooks/hook_audit_cmd.rs similarity index 99% rename from src/hook_audit_cmd.rs rename to src/hooks/hook_audit_cmd.rs index 489eecc0..2138ec93 100644 --- a/src/hook_audit_cmd.rs +++ b/src/hooks/hook_audit_cmd.rs @@ -1,3 +1,5 @@ +//! Audits hook activity logs to show what commands were rewritten and when. + use anyhow::{Context, Result}; use std::collections::HashMap; use std::path::PathBuf; diff --git a/src/hook_check.rs b/src/hooks/hook_check.rs similarity index 98% rename from src/hook_check.rs rename to src/hooks/hook_check.rs index 2716ec15..f84ca7d0 100644 --- a/src/hook_check.rs +++ b/src/hooks/hook_check.rs @@ -1,3 +1,5 @@ +//! Detects whether RTK hooks are installed and warns if they are outdated. + use std::path::PathBuf; const CURRENT_HOOK_VERSION: u8 = 2; diff --git a/src/hook_cmd.rs b/src/hooks/hook_cmd.rs similarity index 98% rename from src/hook_cmd.rs rename to src/hooks/hook_cmd.rs index 29a7365d..8eb4e2fa 100644 --- a/src/hook_cmd.rs +++ b/src/hooks/hook_cmd.rs @@ -1,3 +1,5 @@ +//! Processes incoming hook calls from AI agents and rewrites commands on the fly. + use anyhow::{Context, Result}; use serde_json::{json, Value}; use std::io::{self, Read}; @@ -89,7 +91,7 @@ fn get_rewritten(cmd: &str) -> Option { return None; } - let excluded = crate::config::Config::load() + let excluded = crate::core::config::Config::load() .map(|c| c.hooks.exclude_commands) .unwrap_or_default(); diff --git a/src/init.rs b/src/hooks/init.rs similarity index 99% rename from src/init.rs rename to src/hooks/init.rs index 241a7ef5..87ebf144 100644 --- a/src/init.rs +++ b/src/hooks/init.rs @@ -1,23 +1,25 @@ +//! Sets up RTK hooks so AI coding agents automatically route commands through RTK. + use anyhow::{Context, Result}; use std::fs; use std::io::Write; use std::path::{Path, PathBuf}; use tempfile::NamedTempFile; -use crate::integrity; +use super::integrity; // Embedded hook script (guards before set -euo pipefail) -const REWRITE_HOOK: &str = include_str!("../hooks/rtk-rewrite.sh"); +const REWRITE_HOOK: &str = include_str!("../../hooks/claude/rtk-rewrite.sh"); // Embedded Cursor hook script (preToolUse format) -const CURSOR_REWRITE_HOOK: &str = include_str!("../hooks/cursor-rtk-rewrite.sh"); +const CURSOR_REWRITE_HOOK: &str = include_str!("../../hooks/cursor/rtk-rewrite.sh"); // Embedded OpenCode plugin (auto-rewrite) -const OPENCODE_PLUGIN: &str = include_str!("../hooks/opencode-rtk.ts"); +const OPENCODE_PLUGIN: &str = include_str!("../../hooks/opencode/rtk.ts"); // Embedded slim RTK awareness instructions -const RTK_SLIM: &str = include_str!("../hooks/rtk-awareness.md"); -const RTK_SLIM_CODEX: &str = include_str!("../hooks/rtk-awareness-codex.md"); +const RTK_SLIM: &str = include_str!("../../hooks/claude/rtk-awareness.md"); +const RTK_SLIM_CODEX: &str = include_str!("../../hooks/codex/rtk-awareness.md"); /// Template written by `rtk init` when no filters.toml exists yet. const FILTERS_TEMPLATE: &str = r#"# Project-local RTK filters — commit this file with your repo. @@ -1166,10 +1168,10 @@ fn run_claude_md_mode(global: bool, verbose: u8, install_opencode: bool) -> Resu // ─── Windsurf support ───────────────────────────────────────── /// Embedded Windsurf RTK rules -const WINDSURF_RULES: &str = include_str!("../hooks/windsurf-rtk-rules.md"); +const WINDSURF_RULES: &str = include_str!("../../hooks/windsurf/rules.md"); /// Embedded Cline RTK rules -const CLINE_RULES: &str = include_str!("../hooks/cline-rtk-rules.md"); +const CLINE_RULES: &str = include_str!("../../hooks/cline/rules.md"); // ─── Cline / Roo Code support ───────────────────────────────── @@ -1820,7 +1822,7 @@ fn show_claude_config() -> Result<()> { let has_guards = hook_content.contains("command -v rtk") && hook_content.contains("command -v jq"); let is_thin_delegator = hook_content.contains("rtk rewrite"); - let hook_version = crate::hook_check::parse_hook_version(&hook_content); + let hook_version = super::hook_check::parse_hook_version(&hook_content); if !is_executable { println!( diff --git a/src/integrity.rs b/src/hooks/integrity.rs similarity index 99% rename from src/integrity.rs rename to src/hooks/integrity.rs index 41bcf4e8..96d26368 100644 --- a/src/integrity.rs +++ b/src/hooks/integrity.rs @@ -1,4 +1,4 @@ -//! Hook integrity verification via SHA-256. +//! Detects if someone tampered with the installed hook file. //! //! RTK installs a PreToolUse hook (`rtk-rewrite.sh`) that auto-approves //! rewritten commands with `permissionDecision: "allow"`. Because this diff --git a/src/hooks/mod.rs b/src/hooks/mod.rs new file mode 100644 index 00000000..9c8d451e --- /dev/null +++ b/src/hooks/mod.rs @@ -0,0 +1,10 @@ +//! Hook installation and lifecycle management for AI coding agents. + +pub mod hook_audit_cmd; +pub mod hook_check; +pub mod hook_cmd; +pub mod init; +pub mod integrity; +pub mod rewrite_cmd; +pub mod trust; +pub mod verify_cmd; diff --git a/src/rewrite_cmd.rs b/src/hooks/rewrite_cmd.rs similarity index 90% rename from src/rewrite_cmd.rs rename to src/hooks/rewrite_cmd.rs index 754f51a9..665138f1 100644 --- a/src/rewrite_cmd.rs +++ b/src/hooks/rewrite_cmd.rs @@ -1,3 +1,5 @@ +//! Translates a raw shell command into its RTK-optimized equivalent. + use crate::discover::registry; /// Run the `rtk rewrite` command. @@ -11,7 +13,7 @@ use crate::discover::registry; /// [ "$CMD" = "$REWRITTEN" ] && exit 0 # already RTK, skip /// ``` pub fn run(cmd: &str) -> anyhow::Result<()> { - let excluded = crate::config::Config::load() + let excluded = crate::core::config::Config::load() .map(|c| c.hooks.exclude_commands) .unwrap_or_default(); diff --git a/src/trust.rs b/src/hooks/trust.rs similarity index 99% rename from src/trust.rs rename to src/hooks/trust.rs index c30f977f..5bf6965e 100644 --- a/src/trust.rs +++ b/src/hooks/trust.rs @@ -1,4 +1,4 @@ -//! Trust boundary for project-local TOML filters (SA-2025-RTK-002). +//! Controls which project-local TOML filters are allowed to run. //! //! `.rtk/filters.toml` is loaded from CWD with highest priority. An attacker //! can commit this file to a public repo to control what an LLM sees — hiding @@ -11,7 +11,7 @@ //! - Content changes invalidate trust (re-review required) //! - `RTK_TRUST_PROJECT_FILTERS=1` overrides for CI pipelines -use crate::integrity; +use super::integrity; use anyhow::{Context, Result}; use serde::{Deserialize, Serialize}; use std::collections::HashMap; diff --git a/src/verify_cmd.rs b/src/hooks/verify_cmd.rs similarity index 92% rename from src/verify_cmd.rs rename to src/hooks/verify_cmd.rs index a41f5b90..5412ea26 100644 --- a/src/verify_cmd.rs +++ b/src/hooks/verify_cmd.rs @@ -1,6 +1,8 @@ +//! Runs TOML filter inline tests to make sure filter rules work correctly. + use anyhow::Result; -use crate::toml_filter; +use crate::core::toml_filter; /// Run TOML filter inline tests. /// diff --git a/src/learn/README.md b/src/learn/README.md new file mode 100644 index 00000000..5a277f26 --- /dev/null +++ b/src/learn/README.md @@ -0,0 +1,27 @@ +# Learn — CLI Correction Detection + +> See also [docs/TECHNICAL.md](../../docs/TECHNICAL.md) for the full architecture overview + +## Purpose + +Analyzes Claude Code session history to detect recurring CLI mistakes — commands that fail then get corrected by the agent. Powers the `rtk learn` command, which identifies error patterns (unknown flags, wrong paths, missing args) and can auto-generate `.claude/rules/cli-corrections.md` to prevent them. + +## Key Types + +- **`ErrorType`** — `UnknownFlag`, `CommandNotFound`, `WrongSyntax`, `WrongPath`, `MissingArg`, `PermissionDenied`, `Other(String)` +- **`CorrectionPair`** — Raw detection: wrong command + right command + error output + confidence score +- **`CorrectionRule`** — Deduplicated pattern: wrong pattern + right pattern + occurrence count + base command + +## Dependencies + +- **Uses**: `discover::provider::ClaudeProvider` (session file discovery and command extraction), `lazy_static`/`regex` (error pattern matching), `serde_json` (JSON output) +- **Used by**: `src/main.rs` (routes `rtk learn` command) + +## Detection Algorithm + +1. Extract all commands from JSONL sessions via `ClaudeProvider` +2. Scan chronologically for fail-then-succeed pairs (same base command, first has error output, second succeeds) +3. Classify the error type using regex patterns on the error output +4. Assign confidence scores based on similarity and error clarity +5. Deduplicate into rules (merge identical wrong->right patterns, count occurrences) +6. Filter by `--min-confidence` and `--min-occurrences` thresholds diff --git a/src/learn/detector.rs b/src/learn/detector.rs index 21407668..81ebade8 100644 --- a/src/learn/detector.rs +++ b/src/learn/detector.rs @@ -1,3 +1,5 @@ +//! Pattern-matches CLI errors against known correction rules. + use lazy_static::lazy_static; use regex::Regex; diff --git a/src/learn/mod.rs b/src/learn/mod.rs index 2e1e78b3..28db65cf 100644 --- a/src/learn/mod.rs +++ b/src/learn/mod.rs @@ -1,3 +1,5 @@ +//! Watches for repeated CLI mistakes in coding sessions and suggests corrections. + pub mod detector; pub mod report; diff --git a/src/learn/report.rs b/src/learn/report.rs index 27497406..6ec0b442 100644 --- a/src/learn/report.rs +++ b/src/learn/report.rs @@ -1,3 +1,5 @@ +//! Formats and persists correction suggestions for the user. + use crate::learn::detector::CorrectionRule; use anyhow::Result; use std::collections::HashMap; diff --git a/src/main.rs b/src/main.rs index 0ff5124c..e1c36669 100644 --- a/src/main.rs +++ b/src/main.rs @@ -1,72 +1,27 @@ -mod aws_cmd; -mod binlog; -mod cargo_cmd; -mod cc_economics; -mod ccusage; -mod config; -mod container; -mod curl_cmd; -mod deps; -mod diff_cmd; +mod analytics; +mod cmds; +mod core; mod discover; -mod display_helpers; -mod dotnet_cmd; -mod dotnet_format_report; -mod dotnet_trx; -mod env_cmd; -mod filter; -mod find_cmd; -mod format_cmd; -mod gain; -mod gh_cmd; -mod git; -mod go_cmd; -mod golangci_cmd; -mod grep_cmd; -mod gt_cmd; -mod hook_audit_cmd; -mod hook_check; -mod hook_cmd; -mod init; -mod integrity; -mod json_cmd; +mod hooks; mod learn; -mod lint_cmd; -mod local_llm; -mod log_cmd; -mod ls; -mod mypy_cmd; -mod next_cmd; -mod npm_cmd; mod parser; -mod pip_cmd; -mod playwright_cmd; -mod pnpm_cmd; -mod prettier_cmd; -mod prisma_cmd; -mod psql_cmd; -mod pytest_cmd; -mod rake_cmd; -mod read; -mod rewrite_cmd; -mod rspec_cmd; -mod rubocop_cmd; -mod ruff_cmd; -mod runner; -mod session_cmd; -mod summary; -mod tee; -mod telemetry; -mod toml_filter; -mod tracking; -mod tree; -mod trust; -mod tsc_cmd; -mod utils; -mod verify_cmd; -mod vitest_cmd; -mod wc_cmd; -mod wget_cmd; + +// Re-export command modules for routing +use cmds::cloud::{aws_cmd, container, curl_cmd, psql_cmd, wget_cmd}; +use cmds::dotnet::{binlog, dotnet_cmd, dotnet_format_report, dotnet_trx}; +use cmds::git::{diff_cmd, gh_cmd, git, gt_cmd}; +use cmds::go::{go_cmd, golangci_cmd}; +use cmds::js::{ + lint_cmd, next_cmd, npm_cmd, playwright_cmd, pnpm_cmd, prettier_cmd, prisma_cmd, tsc_cmd, + vitest_cmd, +}; +use cmds::python::{mypy_cmd, pip_cmd, pytest_cmd, ruff_cmd}; +use cmds::ruby::{rake_cmd, rspec_cmd, rubocop_cmd}; +use cmds::rust::{cargo_cmd, runner}; +use cmds::system::{ + deps, env_cmd, find_cmd, format_cmd, grep_cmd, json_cmd, local_llm, log_cmd, ls, read, + summary, tree, wc_cmd, +}; use anyhow::{Context, Result}; use clap::error::ErrorKind; @@ -133,7 +88,7 @@ enum Commands { file: PathBuf, /// Filter: none, minimal, aggressive #[arg(short, long, default_value = "minimal")] - level: filter::FilterLevel, + level: core::filter::FilterLevel, /// Max lines #[arg(short, long, conflicts_with = "tail_lines")] max_lines: Option, @@ -1105,10 +1060,10 @@ fn run_fallback(parse_error: clap::Error) -> Result<()> { } let raw_command = args.join(" "); - let error_message = utils::strip_ansi(&parse_error.to_string()); + let error_message = core::utils::strip_ansi(&parse_error.to_string()); // Start timer before execution to capture actual command runtime - let timer = tracking::TimedExecution::start(); + let timer = core::tracking::TimedExecution::start(); // TOML filter lookup — bypass with RTK_NO_TOML=1 // Use basename of args[0] so absolute paths (/usr/bin/make) still match "^make\b". @@ -1125,12 +1080,12 @@ fn run_fallback(parse_error: clap::Error) -> Result<()> { let toml_match = if std::env::var("RTK_NO_TOML").ok().as_deref() == Some("1") { None } else { - toml_filter::find_matching_filter(&lookup_cmd) + core::toml_filter::find_matching_filter(&lookup_cmd) }; if let Some(filter) = toml_match { // TOML match: capture stdout for filtering - let result = utils::resolved_command(&args[0]) + let result = core::utils::resolved_command(&args[0]) .args(&args[1..]) .stdin(std::process::Stdio::inherit()) .stdout(std::process::Stdio::piped()) // capture @@ -1143,12 +1098,12 @@ fn run_fallback(parse_error: clap::Error) -> Result<()> { // Tee raw output BEFORE filtering on failure — lets LLM re-read if needed let tee_hint = if !output.status.success() { - tee::tee_and_hint(&stdout_raw, &raw_command, output.status.code().unwrap_or(1)) + core::tee::tee_and_hint(&stdout_raw, &raw_command, output.status.code().unwrap_or(1)) } else { None }; - let filtered = toml_filter::apply_filter(filter, &stdout_raw); + let filtered = core::toml_filter::apply_filter(filter, &stdout_raw); println!("{}", filtered); if let Some(hint) = tee_hint { println!("{}", hint); @@ -1160,7 +1115,7 @@ fn run_fallback(parse_error: clap::Error) -> Result<()> { &stdout_raw, &filtered, ); - tracking::record_parse_failure_silent(&raw_command, &error_message, true); + core::tracking::record_parse_failure_silent(&raw_command, &error_message, true); if !output.status.success() { std::process::exit(output.status.code().unwrap_or(1)); @@ -1168,14 +1123,14 @@ fn run_fallback(parse_error: clap::Error) -> Result<()> { } Err(e) => { // Command not found — same behaviour as no-TOML path - tracking::record_parse_failure_silent(&raw_command, &error_message, false); + core::tracking::record_parse_failure_silent(&raw_command, &error_message, false); eprintln!("[rtk: {}]", e); std::process::exit(127); } } } else { // No TOML match: original passthrough behaviour (Stdio::inherit, streaming) - let status = utils::resolved_command(&args[0]) + let status = core::utils::resolved_command(&args[0]) .args(&args[1..]) .stdin(std::process::Stdio::inherit()) .stdout(std::process::Stdio::inherit()) @@ -1186,14 +1141,14 @@ fn run_fallback(parse_error: clap::Error) -> Result<()> { Ok(s) => { timer.track_passthrough(&raw_command, &format!("rtk fallback: {}", raw_command)); - tracking::record_parse_failure_silent(&raw_command, &error_message, true); + core::tracking::record_parse_failure_silent(&raw_command, &error_message, true); if !s.success() { std::process::exit(s.code().unwrap_or(1)); } } Err(e) => { - tracking::record_parse_failure_silent(&raw_command, &error_message, false); + core::tracking::record_parse_failure_silent(&raw_command, &error_message, false); // Command not found or other OS error — single message, no duplicate Clap error eprintln!("[rtk: {}]", e); std::process::exit(127); @@ -1270,7 +1225,7 @@ fn shell_split(input: &str) -> Vec { fn main() -> Result<()> { // Fire-and-forget telemetry ping (1/day, non-blocking) - telemetry::maybe_ping(); + core::telemetry::maybe_ping(); let cli = match Cli::try_parse() { Ok(cli) => cli, @@ -1285,14 +1240,14 @@ fn main() -> Result<()> { // Warn if installed hook is outdated/missing (1/day, non-blocking). // Skip for Gain — it shows its own inline hook warning. if !matches!(cli.command, Commands::Gain { .. }) { - hook_check::maybe_warn(); + hooks::hook_check::maybe_warn(); } // Runtime integrity check for operational commands. // Meta commands (init, gain, verify, config, etc.) skip the check // because they don't go through the hook pipeline. if is_operational_command(&cli.command) { - integrity::runtime_check()?; + hooks::integrity::runtime_check()?; } match cli.command { @@ -1690,19 +1645,19 @@ fn main() -> Result<()> { codex, } => { if show { - init::show_config(codex)?; + hooks::init::show_config(codex)?; } else if uninstall { let cursor = agent == Some(AgentTarget::Cursor); - init::uninstall(global, gemini, codex, cursor, cli.verbose)?; + hooks::init::uninstall(global, gemini, codex, cursor, cli.verbose)?; } else if gemini { let patch_mode = if auto_patch { - init::PatchMode::Auto + hooks::init::PatchMode::Auto } else if no_patch { - init::PatchMode::Skip + hooks::init::PatchMode::Skip } else { - init::PatchMode::Ask + hooks::init::PatchMode::Ask }; - init::run_gemini(global, hook_only, patch_mode, cli.verbose)?; + hooks::init::run_gemini(global, hook_only, patch_mode, cli.verbose)?; } else { let install_opencode = opencode; let install_claude = !opencode; @@ -1711,13 +1666,13 @@ fn main() -> Result<()> { let install_cline = agent == Some(AgentTarget::Cline); let patch_mode = if auto_patch { - init::PatchMode::Auto + hooks::init::PatchMode::Auto } else if no_patch { - init::PatchMode::Skip + hooks::init::PatchMode::Skip } else { - init::PatchMode::Ask + hooks::init::PatchMode::Ask }; - init::run( + hooks::init::run( global, install_claude, install_opencode, @@ -1765,7 +1720,7 @@ fn main() -> Result<()> { format, failures, } => { - gain::run( + analytics::gain::run( project, // added: pass project flag graph, history, @@ -1788,15 +1743,15 @@ fn main() -> Result<()> { all, format, } => { - cc_economics::run(daily, weekly, monthly, all, &format, cli.verbose)?; + analytics::cc_economics::run(daily, weekly, monthly, all, &format, cli.verbose)?; } Commands::Config { create } => { if create { - let path = config::Config::create_default()?; + let path = core::config::Config::create_default()?; println!("Created: {}", path.display()); } else { - config::show_config()?; + core::config::show_config()?; } } @@ -1911,7 +1866,7 @@ fn main() -> Result<()> { } Commands::Session {} => { - session_cmd::run(cli.verbose)?; + analytics::session_cmd::run(cli.verbose)?; } Commands::Learn { @@ -1968,8 +1923,8 @@ fn main() -> Result<()> { } _ => { // Passthrough other prisma subcommands - let timer = tracking::TimedExecution::start(); - let mut cmd = utils::resolved_command("npx"); + let timer = core::tracking::TimedExecution::start(); + let mut cmd = core::utils::resolved_command("npx"); for arg in &args { cmd.arg(arg); } @@ -1985,8 +1940,8 @@ fn main() -> Result<()> { } } } else { - let timer = tracking::TimedExecution::start(); - let status = utils::resolved_command("npx") + let timer = core::tracking::TimedExecution::start(); + let status = core::utils::resolved_command("npx") .arg("prisma") .status() .context("Failed to run npx prisma")?; @@ -2084,21 +2039,21 @@ fn main() -> Result<()> { } Commands::HookAudit { since } => { - hook_audit_cmd::run(since, cli.verbose)?; + hooks::hook_audit_cmd::run(since, cli.verbose)?; } Commands::Hook { command } => match command { HookCommands::Gemini => { - hook_cmd::run_gemini()?; + hooks::hook_cmd::run_gemini()?; } HookCommands::Copilot => { - hook_cmd::run_copilot()?; + hooks::hook_cmd::run_copilot()?; } }, Commands::Rewrite { args } => { let cmd = args.join(" "); - rewrite_cmd::run(&cmd)?; + hooks::rewrite_cmd::run(&cmd)?; } Commands::Proxy { args } => { @@ -2112,7 +2067,7 @@ fn main() -> Result<()> { ); } - let timer = tracking::TimedExecution::start(); + let timer = core::tracking::TimedExecution::start(); // If a single quoted arg contains spaces, split it respecting quotes (#388). // e.g. rtk proxy 'head -50 file.php' → cmd=head, args=["-50", "file.php"] @@ -2139,7 +2094,7 @@ fn main() -> Result<()> { eprintln!("Proxy mode: {} {}", cmd_name, cmd_args.join(" ")); } - let mut child = utils::resolved_command(cmd_name.as_ref()) + let mut child = core::utils::resolved_command(cmd_name.as_ref()) .args(&cmd_args) .stdout(Stdio::piped()) .stderr(Stdio::piped()) @@ -2223,11 +2178,11 @@ fn main() -> Result<()> { } Commands::Trust { list } => { - trust::run_trust(list)?; + hooks::trust::run_trust(list)?; } Commands::Untrust => { - trust::run_untrust()?; + hooks::trust::run_untrust()?; } Commands::Verify { @@ -2236,11 +2191,11 @@ fn main() -> Result<()> { } => { if filter.is_some() { // Filter-specific mode: run only that filter's tests - verify_cmd::run(filter, require_all)?; + hooks::verify_cmd::run(filter, require_all)?; } else { // Default or --require-all: always run integrity check first - integrity::run_verify(cli.verbose)?; - verify_cmd::run(None, require_all)?; + hooks::integrity::run_verify(cli.verbose)?; + hooks::verify_cmd::run(None, require_all)?; } } } diff --git a/src/parser/README.md b/src/parser/README.md index db7b20ed..ddac959d 100644 --- a/src/parser/README.md +++ b/src/parser/README.md @@ -1,5 +1,7 @@ # Parser Infrastructure +> See also [docs/TECHNICAL.md](../../docs/TECHNICAL.md) for the full architecture overview + ## Overview The parser infrastructure provides a unified, three-tier parsing system for tool outputs with graceful degradation: @@ -37,85 +39,13 @@ This ensures RTK **never returns false data silently** while maintaining maximum └─────────────────────────────────────────────────────────┘ ``` -## Usage Example - -### 1. Define Tool-Specific Parser - -```rust -use crate::parser::{OutputParser, ParseResult, TestResult}; - -struct VitestParser; - -impl OutputParser for VitestParser { - type Output = TestResult; - - fn parse(input: &str) -> ParseResult { - // Tier 1: Try JSON parsing - match serde_json::from_str::(input) { - Ok(json) => { - let result = TestResult { - total: json.num_total_tests, - passed: json.num_passed_tests, - failed: json.num_failed_tests, - // ... map fields - }; - ParseResult::Full(result) - } - Err(e) => { - // Tier 2: Try regex extraction - if let Some(stats) = extract_stats_regex(input) { - ParseResult::Degraded( - stats, - vec![format!("JSON parse failed: {}", e)] - ) - } else { - // Tier 3: Passthrough - ParseResult::Passthrough(truncate_output(input, 2000)) - } - } - } - } -} -``` +## Usage Pattern -### 2. Use Parser in Command Module - -```rust -use crate::parser::{OutputParser, TokenFormatter, FormatMode}; - -pub fn run_vitest(args: &[String], verbose: u8) -> Result<()> { - let mut cmd = Command::new("pnpm"); - cmd.arg("vitest").arg("--reporter=json"); - // ... add args - - let output = cmd.output()?; - let stdout = String::from_utf8_lossy(&output.stdout); - - // Parse output - let result = VitestParser::parse(&stdout); - - // Format based on verbosity - let mode = FormatMode::from_verbosity(verbose); - let formatted = match result { - ParseResult::Full(data) => data.format(mode), - ParseResult::Degraded(data, warnings) => { - if verbose > 0 { - for warn in warnings { - eprintln!("[RTK:DEGRADED] {}", warn); - } - } - data.format(mode) - } - ParseResult::Passthrough(raw) => { - eprintln!("[RTK:PASSTHROUGH] Parser failed, showing truncated output"); - raw - } - }; - - println!("{}", formatted); - Ok(()) -} -``` +1. **Implement `OutputParser`** for a tool — try JSON (Tier 1), fall back to regex (Tier 2), then passthrough (Tier 3) +2. **In command module**: call `Parser::parse()`, then `data.format(FormatMode::from_verbosity(verbose))` +3. **Degradation warnings**: print `[RTK:DEGRADED]` in verbose mode, `[RTK:PASSTHROUGH]` on full fallback + +See `src/parser/types.rs` for the `OutputParser` trait and `ParseResult` enum. ## Canonical Types @@ -178,69 +108,11 @@ For build tools (next, webpack, vite, cargo, etc.) ### Existing Module → Parser Trait -**Before:** -```rust -fn run_vitest(args: &[String]) -> Result<()> { - let output = Command::new("vitest").output()?; - let filtered = filter_vitest_output(&output.stdout); - println!("{}", filtered); - Ok(()) -} -``` - -**After:** -```rust -fn run_vitest(args: &[String], verbose: u8) -> Result<()> { - let output = Command::new("vitest") - .arg("--reporter=json") - .output()?; - - let result = VitestParser::parse(&output.stdout); - let mode = FormatMode::from_verbosity(verbose); - - match result { - ParseResult::Full(data) | ParseResult::Degraded(data, _) => { - println!("{}", data.format(mode)); - } - ParseResult::Passthrough(raw) => { - println!("{}", raw); - } - } - Ok(()) -} -``` +Replace direct `filter_*_output()` calls with `Parser::parse()` + `FormatMode`. Key change: add `--reporter=json` flag injection, match on `ParseResult` (Full/Degraded/Passthrough), format with `data.format(mode)`. Degraded and Passthrough tiers handle tool version changes gracefully. ## Testing -### Unit Tests -```bash -cargo test parser::tests -``` - -### Integration Tests -```bash -# Test with real tool outputs -echo '{"testResults": [...]}' | cargo run -- vitest parse -``` - -### Tier Validation -```rust -#[test] -fn test_vitest_json_parsing() { - let json = include_str!("fixtures/vitest-v1.json"); - let result = VitestParser::parse(json); - assert_eq!(result.tier(), 1); // Full parse - assert!(result.is_ok()); -} - -#[test] -fn test_vitest_regex_fallback() { - let text = "Test Files 2 passed (2)\n Tests 13 passed (13)"; - let result = VitestParser::parse(text); - assert_eq!(result.tier(), 2); // Degraded - assert!(!result.warnings().is_empty()); -} -``` +Run `cargo test parser::tests`. Each parser should have tier validation tests: assert `result.tier() == 1` for valid JSON fixtures, `tier() == 2` for regex fallback inputs, and `tier() == 3` for completely malformed output. ## Benefits diff --git a/src/parser/mod.rs b/src/parser/mod.rs index 0af1de19..88ff5ddb 100644 --- a/src/parser/mod.rs +++ b/src/parser/mod.rs @@ -103,7 +103,7 @@ pub trait OutputParser: Sized { /// Truncate output using configured passthrough limit pub fn truncate_passthrough(output: &str) -> String { - let max_chars = crate::config::limits().passthrough_max_chars; + let max_chars = crate::core::config::limits().passthrough_max_chars; truncate_output(output, max_chars) } From 0925cf21e0dd8cfac928ffb4220e8358b2474199 Mon Sep 17 00:00:00 2001 From: aesoft <43991222+aeppling@users.noreply.github.com> Date: Wed, 25 Mar 2026 19:12:46 +0100 Subject: [PATCH 3/5] fix(docs): last review --- ARCHITECTURE.md | 27 ++++------- CLAUDE.md | 2 +- CONTRIBUTING.md | 99 ++++++++++++++------------------------- README.md | 2 +- docs/TECHNICAL.md | 91 +++++++++++++++++++++++++++-------- hooks/README.md | 2 +- src/cmds/README.md | 23 ++++++--- src/cmds/git/README.md | 2 +- src/cmds/system/README.md | 2 +- src/core/README.md | 6 +-- src/discover/README.md | 4 +- src/filters/README.md | 2 +- 12 files changed, 146 insertions(+), 116 deletions(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 334a294b..e13f4e8b 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -176,19 +176,12 @@ Savings by ecosystem: RUST (cmds/rust/) 60-99% cargo test/build/clippy, err ``` -**Total: 67 modules** (45 command modules + 22 infrastructure modules) +### Module Breakdown -### Module Count Breakdown - -- **Command Modules**: 45 in `src/cmds/` (directly exposed to users) -- **Core Infrastructure**: 8 in `src/core/` (utils, filter, tracking, tee, config, toml_filter, display_helpers, telemetry) -- **Hook System**: 8 in `src/hooks/` (init, rewrite, hook_cmd, hook_check, hook_audit, verify, trust, integrity) -- **Analytics**: 4 in `src/analytics/` (gain, cc_economics, ccusage, session_cmd) -- **Git Commands**: 7 operations (status, diff, log, add, commit, push, branch/checkout) -- **JS/TS Tooling**: 8 modules (modern frontend/fullstack development) -- **Python Tooling**: 3 modules (ruff, pytest, pip) -- **Go Tooling**: 2 modules (go test/build/vet, golangci-lint) -- **Ruby Tooling**: 3 modules (rake/minitest, rspec, rubocop) + 1 TOML filter (bundle install) +- **Command Modules**: `src/cmds/` — organized by ecosystem (git, rust, js, python, go, dotnet, cloud, system, ruby). Each ecosystem README lists its files. +- **Core Infrastructure**: `src/core/` — utils, filter, tracking, tee, config, toml_filter, display_helpers, telemetry +- **Hook System**: `src/hooks/` — init, rewrite, hook_cmd, hook_check, hook_audit, verify, trust, integrity +- **Analytics**: `src/analytics/` — gain, cc_economics, ccusage, session_cmd --- @@ -363,7 +356,7 @@ Mirrors: lint, prettier Mirrors: git, cargo ``` ┌────────────────────────────────────────────────────────────────────────┐ -│ Python Commands (3 modules) │ +│ Python Commands │ └────────────────────────────────────────────────────────────────────────┘ Module Strategy Output Format Savings @@ -420,7 +413,7 @@ Commands respect active virtualenv via `sys.executable` paths. ``` ┌────────────────────────────────────────────────────────────────────────┐ -│ Go Commands (2 modules) │ +│ Go Commands │ └────────────────────────────────────────────────────────────────────────┘ Module Strategy Output Format Savings @@ -567,11 +560,11 @@ When adding Python/Go module support: ### Utilities Layer -> For the full utilities API (`truncate`, `strip_ansi`, `execute_command`, `ruby_exec`, etc.), see [src/core/README.md](src/core/README.md). Used by 41 command modules. +> For the full utilities API (`truncate`, `strip_ansi`, `execute_command`, `ruby_exec`, etc.), see [src/core/README.md](src/core/README.md). Used by most command modules. ### Package Manager Detection Pattern -**Critical Infrastructure for JS/TS Stack (8 modules)** +**Critical Infrastructure for JS/TS Stack** ``` ┌────────────────────────────────────────────────────────────────────────┐ @@ -993,7 +986,7 @@ Overhead Sources: ## Extensibility Guide -> For the complete step-by-step process to add a new command (module file, enum variant, routing, tests, documentation), see [CONTRIBUTING.md](CONTRIBUTING.md#complete-contribution-checklist) and the [Standard Module Template in cmds/README.md](src/cmds/README.md). +> For the complete step-by-step process to add a new command (module file, enum variant, routing, tests, documentation), see [src/cmds/README.md — Adding a New Command Filter](src/cmds/README.md#adding-a-new-command-filter). --- diff --git a/CLAUDE.md b/CLAUDE.md index 2ecd1d3c..0dddf14e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -108,7 +108,7 @@ Rust patterns, error handling, and anti-patterns are defined in `.claude/rules/r Testing strategy and performance targets are defined in `.claude/rules/cli-testing.md` (auto-loaded). Key targets: <10ms startup, <5MB memory, 60-90% token savings. -For contribution workflow, design philosophy, and the complete checklist for adding new filters, see [CONTRIBUTING.md](CONTRIBUTING.md). +For contribution workflow and design philosophy, see [CONTRIBUTING.md](CONTRIBUTING.md). For the step-by-step filter implementation checklist, see [src/cmds/README.md](src/cmds/README.md#adding-a-new-command-filter). ## Build Verification (Mandatory) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 479c0ff5..82e69135 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -7,6 +7,7 @@ - [Report an Issue](../../issues/new) - [Open Pull Requests](../../pulls) - [Start a Discussion](../../discussions) +- [Technical Documentation](docs/TECHNICAL.md) — Architecture, end-to-end flow, folder map, how to write tests --- @@ -22,9 +23,9 @@ |------|----------| | **Report** | File a clear issue with steps to reproduce, expected vs actual behavior | | **Fix** | Bug fixes, broken filter repairs | -| **Build** | New filters, new command support, performance improvements | +| **Build** | New filters, new command support, new features (for core features, discuss with maintainers before) | | **Review** | Review open PRs, test changes locally, leave constructive feedback | -| **Document** | Improve docs, add usage examples, clarify existing docs | +| **Document** | Improve docs, clarify | --- ## Design Philosophy @@ -41,7 +42,7 @@ Filters should be flag-aware: default output (no flags) gets aggressively compre ### Transparency -The LLM doesn't know RTK is involved — hooks rewrite commands silently. RTK's output must be a valid, useful subset of the original tool's output, not a different format the LLM wouldn't expect. If an LLM parses `git diff` output, RTK's filtered version must still look like `git diff` output. +The LLM doesn't know RTK is involved for which commands, hooks rewrite commands silently. RTK's output must be a valid, useful subset of the original tool's output, not a different format the LLM wouldn't expect. If an LLM parses `git diff` output, RTK's filtered version must still look like `git diff` output. Don't invent new output formats. Don't add RTK-specific headers or markers in the default output. The filtered output should be indistinguishable from "a shorter version of the real command." @@ -57,12 +58,15 @@ Every filter needs a fallback path. Every hook must handle malformed input grace `lazy_static!` for all regex. No network calls. No disk reads in the hot path. Benchmark before/after with `hyperfine`. +### Extensibility + +Always use components already in place to avoid duplication, also use extensible modules when this is possible. +If you want to submit a new core feature, this is an important point to watch. + --- ## What Belongs in RTK? -RTK filters **development CLI commands** consumed by LLM coding assistants — the commands an AI agent runs during a coding session: test runners, linters, build tools, VCS operations, package managers, file operations. - ### In Scope Commands that produce **text output** (typically 100+ tokens) and can be compressed **60%+** without losing essential information for the LLM. @@ -75,12 +79,15 @@ Commands that produce **text output** (typically 100+ tokens) and can be compres - File operations (ls, tree, grep, find, cat/head/tail) - Infrastructure tools with text output (docker, kubectl, terraform) +When implementing a new filter/cmds, be aware of the [Design Philosophy](#design-philosophy) above. + ### Out of Scope -- Interactive TUIs (htop, vim, less) — not batch-mode compatible -- Binary output (images, compiled artifacts) — no text to filter -- Trivial commands (<100 tokens typical output) — not worth the overhead -- Commands with no text output — nothing to compress +- Interactive TUIs (htop, vim, less): not batch-mode compatible +- Binary output (images, compiled artifacts): no text to filter +- Trivial commands: not worth the overhead and may loose important informations +- Commands with no text output: nothing to compress +- Others features not related to a LLM-proxy like RTK ### TOML vs Rust: Which One? @@ -94,20 +101,9 @@ Commands that produce **text output** (typically 100+ tokens) and can be compres See [`src/filters/README.md`](src/filters/README.md) for TOML filter guidance and [`src/cmds/README.md`](src/cmds/README.md) for Rust module guidance. -### Complete Contribution Checklist - -Adding a new filter or command requires changes in multiple places: +### Adding a Filter -1. **Create the filter** — TOML file in `src/filters/` or Rust module in `src/cmds//` -2. **Add rewrite pattern** — Entry in `src/discover/rules.rs` (PATTERNS + RULES arrays at matching index) so hooks auto-rewrite the command -3. **Register in main.rs** — (Rust modules only) Three changes: - - Add `pub mod mymod;` to the ecosystem's `mod.rs` (e.g., `src/cmds/system/mod.rs`) - - Add variant to `Commands` enum in `main.rs` with `#[arg(trailing_var_arg = true, allow_hyphen_values = true)]` - - Add routing match arm in `main.rs` to call `mymod::run()` -4. **Write tests** — Real fixture, snapshot test, token savings >= 60% -5. **Update docs** — README.md command list, CHANGELOG.md - -See [src/cmds/README.md](src/cmds/README.md#common-pattern) for the standard module template with timer, fallback, tee, and tracking. +For the step-by-step checklist (create filter, register rewrite pattern, register in main.rs, write tests, update docs), see [src/cmds/README.md — Adding a New Command Filter](src/cmds/README.md#adding-a-new-command-filter). --- @@ -141,12 +137,13 @@ chore(proxy): remove-deprecated-flags **Each PR must focus on a single feature, fix, or change.** The diff must stay in-scope with the description written by the author in the PR title and body. Out-of-scope changes (unrelated refactors, drive-by fixes, formatting of untouched files) must go in a separate PR. **For large features or refactors**, prefer multi-part PRs over one enormous PR. Split the work into logical, reviewable chunks that can each be merged independently. Examples: -- Part 1: Add data model and tests -- Part 2: Add CLI command and integration -- Part 3: Update documentation and CHANGELOG +- feat(Part 1): Add data model and tests +- feat(Part 2): Add CLI command and integration +- feat(Part 3): Update documentation and CHANGELOG **Why**: Small, focused PRs are easier to review, safer to merge, and faster to ship. Large PRs slow down review, hide bugs, and increase merge conflict risk. + ### 1. Create Your Branch ```bash @@ -163,7 +160,7 @@ git checkout -b "feat(scope): your-clear-description" **No obvious comments.** Don't comment what the code already says. Comments should explain *why*, never *what* to avoid noise. -**Large command files are expected.** Command modules (`*_cmd.rs`) contain the implementation, tests, and fixture in the same file. A big file is fine when it's self-contained for one command. +**Large command files are expected.** Command modules (`*_cmd.rs`) contain the implementation, tests, and fixture in the same file. A big file is fine when it's self-contained for one command. This will be moved in the future. ### 3. Add Tests @@ -208,6 +205,8 @@ your branch --> develop (review + CI + integration testing) --> version branch - Every change **must** include tests. We follow **TDD (Red-Green-Refactor)**: write a failing test first, implement the minimum to pass, then refactor. +For how to write tests (fixtures, snapshots, token savings verification), see [docs/TECHNICAL.md — Testing](docs/TECHNICAL.md#testing). + ### Test Types | Type | Where | Run With | @@ -217,36 +216,6 @@ Every change **must** include tests. We follow **TDD (Red-Green-Refactor)**: wri | **Smoke tests** | `scripts/test-all.sh` (69 assertions) | `bash scripts/test-all.sh` | | **Integration tests** | `#[ignore]` tests requiring installed binary | `cargo test --ignored` | -### How to Write Tests - -Tests for new commands live **in the module file itself** inside a `#[cfg(test)] mod tests` block (e.g. tests for `src/cmds/cloud/container.rs` go at the bottom of that same file). - -**1. Create a fixture from real command output** (not synthetic data): -```bash -kubectl get pods > tests/fixtures/kubectl_pods_raw.txt -``` - -**2. Write your test in the same module file** (`#[cfg(test)] mod tests`): -```rust -#[test] -fn test_my_filter() { - let input = include_str!("../tests/fixtures/my_cmd_raw.txt"); - let output = filter_my_cmd(input); - assert_snapshot!(output); -} -``` - -**3. Verify token savings**: -```rust -#[test] -fn test_my_filter_savings() { - let input = include_str!("../tests/fixtures/my_cmd_raw.txt"); - let output = filter_my_cmd(input); - let savings = 100.0 - (count_tokens(&output) as f64 / count_tokens(input) as f64 * 100.0); - assert!(savings >= 60.0, "Expected >=60% savings, got {:.1}%", savings); -} -``` - ### Pre-Commit Gate (mandatory) All three must pass before any PR: @@ -268,15 +237,19 @@ cargo fmt --all --check && cargo clippy --all-targets && cargo test ## Documentation -Every change **must** include documentation updates. Update the relevant file(s) depending on what you changed: +Every change **must** include documentation updates. Use this table to find which docs to update: -| What you changed | Update | -|------------------|--------| -| New command or filter | [README.md](README.md) (command list + examples) and [CHANGELOG.md](CHANGELOG.md) | -| Architecture or internal design | [ARCHITECTURE.md](ARCHITECTURE.md) | -| Installation or setup | [INSTALL.md](INSTALL.md) | +| What you changed | Update these docs | +|------------------|-------------------| +| New Rust filter (`src/cmds/`) | Ecosystem `README.md` (e.g., `src/cmds/git/README.md`), [README.md](README.md) command list, [CHANGELOG.md](CHANGELOG.md) | +| New TOML filter (`src/filters/`) | [src/filters/README.md](src/filters/README.md) if naming conventions change, [README.md](README.md) command list, [CHANGELOG.md](CHANGELOG.md) | +| New rewrite pattern | `src/discover/rules.rs` — see [Adding a New Command Filter](src/cmds/README.md#adding-a-new-command-filter) | +| Core infrastructure (`src/core/`) | [src/core/README.md](src/core/README.md), [docs/TECHNICAL.md](docs/TECHNICAL.md) if flow changes | +| Hook system (`src/hooks/`) | [src/hooks/README.md](src/hooks/README.md), [hooks/README.md](hooks/README.md) for agent-facing docs | +| Architecture or design change | [ARCHITECTURE.md](ARCHITECTURE.md), [docs/TECHNICAL.md](docs/TECHNICAL.md) | | Bug fix or breaking change | [CHANGELOG.md](CHANGELOG.md) | -| Tracking / analytics | [docs/tracking.md](docs/tracking.md) | + +**Navigation**: [CONTRIBUTING.md](CONTRIBUTING.md) (you are here) → [docs/TECHNICAL.md](docs/TECHNICAL.md) (architecture + flow) → each folder's `README.md` (implementation details). Keep documentation concise and practical -- examples over explanations. diff --git a/README.md b/README.md index 7401256d..2b6a9495 100644 --- a/README.md +++ b/README.md @@ -33,7 +33,7 @@ --- -rtk filters and compresses command outputs before they reach your LLM context. Single Rust binary, zero dependencies, <10ms overhead. +rtk filters and compresses command outputs before they reach your LLM context. Single Rust binary, 100+ supported commands, <10ms overhead. ## Token Savings (30-min Claude Code Session) diff --git a/docs/TECHNICAL.md b/docs/TECHNICAL.md index 18e33b71..541f712d 100644 --- a/docs/TECHNICAL.md +++ b/docs/TECHNICAL.md @@ -1,9 +1,10 @@ # RTK Technical Documentation -> **Start here** for a guided tour of how RTK works end-to-end. For the deep reference (filtering taxonomy, performance benchmarks, architecture decisions), see [ARCHITECTURE.md](../ARCHITECTURE.md). +> **Start here** for a guided tour of how RTK works end-to-end. > -> See `CLAUDE.md` for development commands and coding guidelines. -> Each folder has its own README.md with implementation details, file descriptions, and contribution guidelines. +> - [CONTRIBUTING.md](../CONTRIBUTING.md) — Design philosophy, PR process, branch naming, testing requirements +> - [ARCHITECTURE.md](../ARCHITECTURE.md) — Deep reference: filtering taxonomy, performance benchmarks, architecture decisions +> - Each folder has its own `README.md` with implementation details and file descriptions --- @@ -11,7 +12,7 @@ LLM-powered coding agents (Claude Code, Copilot, Cursor, etc.) consume tokens for every CLI command output they process. Most command outputs contain boilerplate, progress bars, ANSI escape codes, and verbose formatting that wastes tokens without providing actionable information. -RTK sits between the agent and the CLI, filtering outputs to keep only what matters. This achieves 60-90% token savings per command, reducing costs and increasing effective context window utilization. RTK is a single Rust binary with zero external dependencies, adding less than 10ms overhead per command. +RTK sits between the agent and the CLI, filtering outputs to keep only what matters. This achieves 60-90% token savings per command, reducing costs and increasing effective context window utilization. RTK is a single Rust binary with no runtime dependencies beyond the compiled binary itself, adding less than 10ms overhead per command. --- @@ -85,7 +86,7 @@ When an LLM agent runs a command (e.g., `git status`): 1. The agent fires a `PreToolUse` event (or equivalent) containing the command as JSON 2. The hook script reads the JSON, extracts the command string 3. The hook calls `rtk rewrite "git status"` as a subprocess -4. `rtk rewrite` consults the command registry (70+ patterns) and returns `rtk git status` +4. `rtk rewrite` consults the command registry and returns `rtk git status` 5. The hook sends a response telling the agent to use the rewritten command 6. If anything fails (jq missing, rtk not found, no match), the hook exits silently -- the raw command runs unchanged @@ -98,7 +99,7 @@ All rewrite logic lives in Rust (`src/discover/registry.rs`). Hooks are thin del Once the rewritten command reaches RTK: 1. **Telemetry**: `telemetry::maybe_ping()` fires a non-blocking daily usage ping -2. **Clap parsing**: `Cli::try_parse()` matches against the `Commands` enum (~50 subcommands) +2. **Clap parsing**: `Cli::try_parse()` matches against the `Commands` enum 3. **Hook check**: `hook_check::maybe_warn()` warns if the installed hook is outdated (rate-limited to 1/day) 4. **Integrity check**: `integrity::runtime_check()` verifies the hook's SHA-256 hash for operational commands 5. **Routing**: A `match cli.command` dispatches to the specialized filter module @@ -109,9 +110,9 @@ If Clap parsing fails (command not in the enum), the fallback path runs instead. RTK has two filter systems: -**Rust Filters** (~45 commands): Compiled modules in `src/cmds/` that execute the command, parse its output, and apply specialized transformations (regex, JSON, state machines). +**Rust Filters**: Compiled modules in `src/cmds/` that execute the command, parse its output, and apply specialized transformations (regex, JSON, state machines). -**TOML DSL Filters** (~60 built-in): Declarative filters in `src/filters/*.toml` that apply regex-based line filtering, truncation, and section extraction. Applied in `run_fallback()` when no Rust filter matches. +**TOML DSL Filters**: Declarative filters in `src/filters/*.toml` that apply regex-based line filtering, truncation, and section extraction. Applied in `run_fallback()` when no Rust filter matches. Each filter module follows the same pattern: 1. Start a timer (`TimedExecution::start()`) @@ -180,14 +181,14 @@ Start here, then drill down into each README for file-level details. | Directory | What it does | What you'll find in its README | |-----------|-------------|-------------------------------| | `main.rs` | CLI entry point, `Commands` enum, routing match | _(no README — read the file directly)_ | -| [`core/`](../src/core/README.md) | Shared infrastructure (8 files) | Tracking DB schema, config system, tee recovery, TOML filter engine, utility functions | -| [`hooks/`](../src/hooks/README.md) | Hook system (8 files) | Installation flow (`rtk init`), integrity verification, rewrite command, trust model | -| [`analytics/`](../src/analytics/README.md) | Token savings analytics (4 files) | `rtk gain` dashboard, Claude Code economics, ccusage parsing | -| [`cmds/`](../src/cmds/README.md) | **Command filters (45 files, 9 ecosystems)** | Common filter pattern, cross-command routing, token savings table, **links to each ecosystem** | -| [`discover/`](../src/discover/README.md) | History analysis + rewrite registry (5 files) | 70+ rewrite patterns, session providers, compound command splitting | -| [`learn/`](../src/learn/README.md) | CLI correction detection (3 files) | Error classification (6 types), correction pair detection, rule generation | -| [`parser/`](../src/parser/README.md) | Parser infrastructure (4 files) | Canonical types (TestResult, LintResult, etc.), 3-tier format modes, migration guide | -| [`filters/`](../src/filters/README.md) | 60+ TOML filter configs | TOML DSL syntax, 8-stage pipeline, inline testing, naming conventions | +| [`core/`](../src/core/README.md) | Shared infrastructure | Tracking DB schema, config system, tee recovery, TOML filter engine, utility functions | +| [`hooks/`](../src/hooks/README.md) | Hook system | Installation flow (`rtk init`), integrity verification, rewrite command, trust model | +| [`analytics/`](../src/analytics/README.md) | Token savings analytics | `rtk gain` dashboard, Claude Code economics, ccusage parsing | +| [`cmds/`](../src/cmds/README.md) | **Command filters (9 ecosystems)** | Common filter pattern, cross-command routing, token savings table, **links to each ecosystem** | +| [`discover/`](../src/discover/README.md) | History analysis + rewrite registry | Rewrite patterns, session providers, compound command splitting | +| [`learn/`](../src/learn/README.md) | CLI correction detection | Error classification, correction pair detection, rule generation | +| [`parser/`](../src/parser/README.md) | Parser infrastructure | Canonical types (TestResult, LintResult, etc.), 3-tier format modes, migration guide | +| [`filters/`](../src/filters/README.md) | TOML filter configs | TOML DSL syntax, 8-stage pipeline, inline testing, naming conventions | ### `hooks/` — Deployed hook artifacts (root directory) @@ -206,7 +207,7 @@ Start here, then drill down into each README for file-level details. ## 5. Hook System Summary -RTK supports 7 LLM agents through hook integrations: +RTK supports the following LLM agents through hook integrations: | Agent | Hook Type | Mechanism | Can Modify Command? | |-------|-----------|-----------|---------------------| @@ -228,7 +229,7 @@ RTK supports 7 LLM agents through hook integrations: ### Rust Filters (cmds/**) -Compiled filter modules for complex transformations. ~45 commands with 60-95% token savings. +Compiled filter modules for complex transformations with 60-95% token savings. > **Details**: [`src/cmds/README.md`](../src/cmds/README.md) and each ecosystem subdirectory README. @@ -257,7 +258,59 @@ Achieved through: --- -## 8. Future Improvements +## 8. Testing + +Tests live **in the module file itself** inside a `#[cfg(test)] mod tests` block (e.g., tests for `src/cmds/cloud/container.rs` go at the bottom of that same file). + +### How to Write Tests + +**1. Create a fixture from real command output** (not synthetic data): +```bash +kubectl get pods > tests/fixtures/kubectl_pods_raw.txt +``` + +**2. Write your test in the same module file** (`#[cfg(test)] mod tests`): +```rust +#[test] +fn test_my_filter() { + let input = include_str!("../tests/fixtures/my_cmd_raw.txt"); + let output = filter_my_cmd(input); + assert!(output.contains("expected content")); + assert!(!output.contains("noise line")); +} +``` + +**3. Verify token savings** (60% minimum required): +```rust +#[test] +fn test_my_filter_savings() { + let input = include_str!("../tests/fixtures/my_cmd_raw.txt"); + let output = filter_my_cmd(input); + let savings = 100.0 - (count_tokens(&output) as f64 / count_tokens(input) as f64 * 100.0); + assert!(savings >= 60.0, "Expected >=60% savings, got {:.1}%", savings); +} +``` + +### Test Organization + +``` +tests/ +├── fixtures/ # Real command output (never synthetic) +│ ├── git_log_raw.txt +│ ├── cargo_test_raw.txt +│ └── dotnet/ # Ecosystem-specific fixtures +└── integration_test.rs # Integration tests (#[ignore]) +``` + +- **Unit tests**: `#[cfg(test)] mod tests` embedded in each module +- **Fixtures**: real command output in `tests/fixtures/` +- **Integration tests**: `#[ignore]` attribute, run with `cargo test --ignored` + +> For testing requirements, pre-commit gate, and PR checklist, see [CONTRIBUTING.md — Testing](../CONTRIBUTING.md#testing). + +--- + +## 9. Future Improvements - **Extract cli.rs**: Move `Commands` enum, 13 sub-enums (`GitCommands`, `CargoCommands`, etc.), and `AgentTarget` from main.rs to a dedicated cli.rs module. This would reduce main.rs from ~2600 to ~1500 lines. - **Split routing**: Extract the `match cli.command { ... }` block into a separate routing module. diff --git a/hooks/README.md b/hooks/README.md index 72ee98c6..9d5e6380 100644 --- a/hooks/README.md +++ b/hooks/README.md @@ -158,7 +158,7 @@ if (rewritten && rewritten !== command) { ## Command Rewrite Registry -The registry (`src/discover/registry.rs`) handles 70+ command patterns across these categories: +The registry (`src/discover/registry.rs`) handles command patterns across these categories: | Category | Examples | Savings | |----------|----------|---------| diff --git a/src/cmds/README.md b/src/cmds/README.md index 4577cd15..a84e8e74 100644 --- a/src/cmds/README.md +++ b/src/cmds/README.md @@ -16,7 +16,7 @@ Rust modules exist here because they need capabilities TOML filters don't have: **Ecosystem placement**: Match the command's language/toolchain. Use `system/` for language-agnostic commands. New ecosystem when 3+ related commands justify it. -For the full contribution checklist (including `discover/rules.rs` registration), see [CONTRIBUTING.md](../../CONTRIBUTING.md#complete-contribution-checklist). +For the full contribution checklist (including `discover/rules.rs` registration), see [Adding a New Command Filter](#adding-a-new-command-filter) below. ## Purpose All command-specific filter modules that execute CLI commands and transform their output to minimize LLM token consumption. Each module follows a consistent pattern: execute the underlying command, filter its output through specialized parsers, track token savings, and propagate exit codes. @@ -81,7 +81,7 @@ Six phases: **timer** → **execute** → **filter (with fallback)** → **tee o - `lint_cmd` routes to `mypy_cmd` or `ruff_cmd` when detecting Python projects - `format_cmd` routes to `prettier_cmd` or `ruff_cmd` depending on the formatter detected -- `gh_cmd` imports markdown filtering helpers from `git` +- `gh_cmd` imports `compact_diff()` from `git` for diff formatting (markdown helpers are defined in `gh_cmd` itself) ## Cross-Cutting Behavior Contracts @@ -120,10 +120,10 @@ All modules accept `verbose: u8`. Use it to print debug info (command being run, **Filter passthrough** — silent passthrough, no warning: - `gh_cmd.rs`, `pip_cmd.rs`, `container.rs`, `dotnet_cmd.rs` — `run_passthrough()` skips filtering without warning -- `pnpm_cmd.rs`, `playwright_cmd.rs` — 3-tier degradation but no tee recovery on final tier +- `pnpm_cmd.rs` — 3-tier degradation but no tee recovery on final tier -**Tee recovery** — currently 12/38 modules implement tee. Missing from high-risk modules: -- `pnpm_cmd.rs`, `playwright_cmd.rs` — 3-tier parsers, no tee +**Tee recovery** — missing from some high-risk modules: +- `pnpm_cmd.rs` — 3-tier parser, no tee - `gh_cmd.rs` — aggressive markdown filtering, no tee - `ruff_cmd.rs`, `golangci_cmd.rs` — JSON parsers, no tee - `psql_cmd.rs` — has tee but exits before calling it on error path @@ -140,4 +140,15 @@ All modules accept `verbose: u8`. Use it to print debug info (command being run, ## Adding a New Command Filter -Follow the [Common Pattern](#common-pattern) above (timer, execute, filter with fallback, tee, track, exit code). For the full step-by-step checklist, see [CONTRIBUTING.md](../../CONTRIBUTING.md#complete-contribution-checklist). For the Rust module structure, see [`.claude/rules/rust-patterns.md`](../../.claude/rules/rust-patterns.md). +Adding a new filter or command requires changes in multiple places: + +1. **Create the filter** — TOML file in [`src/filters/`](../filters/README.md) or Rust module in `src/cmds//` +2. **Add rewrite pattern** — Entry in `src/discover/rules.rs` (PATTERNS + RULES arrays at matching index) so hooks auto-rewrite the command +3. **Register in main.rs** — (Rust modules only) Three changes: + - Add `pub mod mymod;` to the ecosystem's `mod.rs` (e.g., `src/cmds/system/mod.rs`) + - Add variant to `Commands` enum in `main.rs` with `#[arg(trailing_var_arg = true, allow_hyphen_values = true)]` + - Add routing match arm in `main.rs` to call `mymod::run()` +4. **Write tests** — Real fixture, snapshot test, token savings >= 60% (see [testing rules](../../.claude/rules/cli-testing.md)) +5. **Update docs** — README.md command list, CHANGELOG.md + +Follow the [Common Pattern](#common-pattern) above for the module template (timer, fallback, tee, tracking, exit code). For TOML-vs-Rust decision criteria, see [CONTRIBUTING.md](../../CONTRIBUTING.md#toml-vs-rust-which-one). diff --git a/src/cmds/git/README.md b/src/cmds/git/README.md index 6b8dc570..ec595ca0 100644 --- a/src/cmds/git/README.md +++ b/src/cmds/git/README.md @@ -11,5 +11,5 @@ ## Cross-command -- `gh_cmd.rs` imports markdown filtering helpers from `git.rs` for PR body rendering +- `gh_cmd.rs` imports `compact_diff()` from `git.rs` for diff formatting; markdown helpers (`filter_markdown_body`, `filter_markdown_segment`) are defined in `gh_cmd.rs` itself - `diff_cmd.rs` is a standalone ultra-condensed diff (separate from `git diff`) diff --git a/src/cmds/system/README.md b/src/cmds/system/README.md index bffdff17..ec3b327b 100644 --- a/src/cmds/system/README.md +++ b/src/cmds/system/README.md @@ -7,7 +7,7 @@ - `read.rs` uses `core/filter` for language-aware code stripping (FilterLevel: none/minimal/aggressive) - `grep_cmd.rs` reads `core/config` for `limits.grep_max_results` and `limits.grep_max_per_file` - `local_llm.rs` (`rtk smart`) uses `core/filter` for heuristic file summarization -- `format_cmd.rs` is a cross-ecosystem dispatcher: auto-detects and routes to `prettier_cmd`, `ruff_cmd`, or `black` +- `format_cmd.rs` is a cross-ecosystem dispatcher: auto-detects and routes to `prettier_cmd` or `ruff_cmd` (black is handled inline, not as a separate module) ## Cross-command diff --git a/src/core/README.md b/src/core/README.md index 2f0a4684..29befe0c 100644 --- a/src/core/README.md +++ b/src/core/README.md @@ -29,7 +29,7 @@ The TOML DSL applies 8 stages in order: Three-tier filter lookup (first match wins): 1. `.rtk/filters.toml` (project-local, requires `rtk trust`) 2. `~/.config/rtk/filters.toml` (user-global) -3. Built-in filters concatenated by `build.rs` at compile time (57+ filters) +3. Built-in filters concatenated by `build.rs` at compile time ## Tracking Database Schema @@ -94,7 +94,7 @@ passthrough_max_chars = 2000 ## Shared Utilities (utils.rs) -Key functions available to all command modules (41 modules depend on `core::utils`): +Key functions available to all command modules: | Function | Purpose | |----------|---------| @@ -123,7 +123,7 @@ Consumers that parse structured output (JSON, NDJSON, state machines) should cal - `ls.rs`, `tree.rs` — exit before `track()` on error path (lost metrics) - `container.rs` — inconsistent tracking across subcommands -- 26/38 command modules missing tee integration — see `src/cmds/README.md` for the full list +- Many command modules still missing tee integration — see `src/cmds/README.md` for the full list ## Adding New Functionality Place new infrastructure code here if it meets **all** of these criteria: (1) it has no dependencies on command modules or hooks, (2) it is used by two or more other modules, and (3) it provides a general-purpose utility rather than command-specific logic. Follow the existing pattern of lazy-initialized resources (`lazy_static!` for regex, on-demand config loading) to preserve the <10ms startup target. Add `#[cfg(test)] mod tests` with unit tests in the same file. diff --git a/src/discover/README.md b/src/discover/README.md index 116d059b..cb523f4d 100644 --- a/src/discover/README.md +++ b/src/discover/README.md @@ -6,7 +6,7 @@ Scans Claude Code JSONL session files to identify commands that could benefit from RTK filtering. Powers the `rtk discover` command, which reports missed savings opportunities and adoption metrics. -Also provides the **command rewrite registry** — the single source of truth for all 70+ patterns used by every LLM agent hook to decide which commands to rewrite. +Also provides the **command rewrite registry** — the single source of truth for all rewrite patterns used by every LLM agent hook to decide which commands to rewrite. ## Key Types @@ -22,7 +22,7 @@ Also provides the **command rewrite registry** — the single source of truth fo ## Registry Architecture -`registry.rs` (2270 lines) is the largest file in the project. It contains: +`registry.rs` is the largest file in the project. It contains: 1. **Pattern matching** — Compiled regexes in `lazy_static!` matching command prefixes (e.g., `^git\s+(status|log|diff|...)`) 2. **Compound splitting** — `split_command_chain()` handles `&&`, `||`, `;`, `|`, `&` operators with shell quoting awareness diff --git a/src/filters/README.md b/src/filters/README.md index e6bf7453..1fabae7a 100644 --- a/src/filters/README.md +++ b/src/filters/README.md @@ -15,7 +15,7 @@ TOML works well for commands with **predictable, line-by-line text output** wher - Simple linters (shellcheck, yamllint, hadolint) — strip context, keep findings - Infra tools (terraform plan, helm, rsync) — strip progress, keep summary -For the full contribution checklist (including `discover/rules.rs` registration), see [CONTRIBUTING.md](../../CONTRIBUTING.md#complete-contribution-checklist). +For the full contribution checklist (including `discover/rules.rs` registration), see [src/cmds/README.md — Adding a New Command Filter](../cmds/README.md#adding-a-new-command-filter). ## Adding a filter From 135fea693e9cd0949226178e3e5823e7a71ad82a Mon Sep 17 00:00:00 2001 From: aesoft <43991222+aeppling@users.noreply.github.com> Date: Wed, 25 Mar 2026 19:56:53 +0100 Subject: [PATCH 4/5] fix(refacto): conflict w/ dev --- src/analytics/gain.rs | 2 +- src/analytics/session_cmd.rs | 2 +- src/cmds/cloud/aws_cmd.rs | 2 +- src/cmds/cloud/curl_cmd.rs | 2 +- src/cmds/dotnet/dotnet_cmd.rs | 4 ++-- src/cmds/git/gh_cmd.rs | 2 +- src/cmds/js/lint_cmd.rs | 4 ++-- src/cmds/js/vitest_cmd.rs | 4 ++-- src/cmds/ruby/rake_cmd.rs | 5 ++++- src/cmds/rust/cargo_cmd.rs | 3 ++- src/cmds/system/format_cmd.rs | 4 ++-- src/core/toml_filter.rs | 10 ++++++---- src/hooks/mod.rs | 1 + src/hooks/rewrite_cmd.rs | 2 +- 14 files changed, 27 insertions(+), 20 deletions(-) diff --git a/src/analytics/gain.rs b/src/analytics/gain.rs index dd3a3346..3938334a 100644 --- a/src/analytics/gain.rs +++ b/src/analytics/gain.rs @@ -1,9 +1,9 @@ //! Shows users how many tokens RTK has saved them over time. use crate::core::display_helpers::{format_duration, print_period_table}; -use crate::hooks::hook_check; use crate::core::tracking::{DayStats, MonthStats, Tracker, WeekStats}; use crate::core::utils::format_tokens; +use crate::hooks::hook_check; use anyhow::{Context, Result}; use colored::Colorize; use serde::Serialize; diff --git a/src/analytics/session_cmd.rs b/src/analytics/session_cmd.rs index 364bbe9e..c36a42c2 100644 --- a/src/analytics/session_cmd.rs +++ b/src/analytics/session_cmd.rs @@ -1,8 +1,8 @@ //! Compares RTK-routed vs raw commands in a coding session. +use crate::core::utils::format_tokens; use crate::discover::provider::{ClaudeProvider, ExtractedCommand, SessionProvider}; use crate::discover::registry::{classify_command, split_command_chain, Classification}; -use crate::core::utils::format_tokens; use anyhow::{Context, Result}; use std::fs; use std::path::PathBuf; diff --git a/src/cmds/cloud/aws_cmd.rs b/src/cmds/cloud/aws_cmd.rs index 8b701f26..bb1757ec 100644 --- a/src/cmds/cloud/aws_cmd.rs +++ b/src/cmds/cloud/aws_cmd.rs @@ -3,9 +3,9 @@ //! Replaces verbose `--output table`/`text` with JSON, then compresses. //! Specialized filters for high-frequency commands (STS, S3, EC2, ECS, RDS, CloudFormation). -use crate::json_cmd; use crate::core::tracking; use crate::core::utils::{join_with_overflow, resolved_command, truncate_iso_date}; +use crate::json_cmd; use anyhow::{Context, Result}; use serde_json::Value; diff --git a/src/cmds/cloud/curl_cmd.rs b/src/cmds/cloud/curl_cmd.rs index 046ae0da..7141ad72 100644 --- a/src/cmds/cloud/curl_cmd.rs +++ b/src/cmds/cloud/curl_cmd.rs @@ -1,8 +1,8 @@ //! Runs curl and auto-compresses JSON responses. -use crate::json_cmd; use crate::core::tracking; use crate::core::utils::{resolved_command, truncate}; +use crate::json_cmd; use anyhow::{Context, Result}; pub fn run(args: &[String], verbose: u8) -> Result<()> { diff --git a/src/cmds/dotnet/dotnet_cmd.rs b/src/cmds/dotnet/dotnet_cmd.rs index 5f9f9b71..5f050883 100644 --- a/src/cmds/dotnet/dotnet_cmd.rs +++ b/src/cmds/dotnet/dotnet_cmd.rs @@ -1,10 +1,10 @@ //! Filters dotnet CLI output — build, test, and format results. use crate::binlog; -use crate::dotnet_format_report; -use crate::dotnet_trx; use crate::core::tracking; use crate::core::utils::{resolved_command, truncate}; +use crate::dotnet_format_report; +use crate::dotnet_trx; use anyhow::{Context, Result}; use quick_xml::events::Event; use quick_xml::Reader; diff --git a/src/cmds/git/gh_cmd.rs b/src/cmds/git/gh_cmd.rs index 5847775b..e008a2f1 100644 --- a/src/cmds/git/gh_cmd.rs +++ b/src/cmds/git/gh_cmd.rs @@ -3,9 +3,9 @@ //! Provides token-optimized alternatives to verbose `gh` commands. //! Focuses on extracting essential information from JSON outputs. -use crate::git; use crate::core::tracking; use crate::core::utils::{ok_confirmation, resolved_command, truncate}; +use crate::git; use anyhow::{Context, Result}; use lazy_static::lazy_static; use regex::Regex; diff --git a/src/cmds/js/lint_cmd.rs b/src/cmds/js/lint_cmd.rs index bb384bae..e7a88e89 100644 --- a/src/cmds/js/lint_cmd.rs +++ b/src/cmds/js/lint_cmd.rs @@ -1,10 +1,10 @@ //! Filters ESLint and Biome linter output, grouping violations by rule. use crate::core::config; -use crate::mypy_cmd; -use crate::ruff_cmd; use crate::core::tracking; use crate::core::utils::{package_manager_exec, resolved_command, truncate}; +use crate::mypy_cmd; +use crate::ruff_cmd; use anyhow::{Context, Result}; use serde::{Deserialize, Serialize}; use std::collections::HashMap; diff --git a/src/cmds/js/vitest_cmd.rs b/src/cmds/js/vitest_cmd.rs index 74d09206..bc68e2dd 100644 --- a/src/cmds/js/vitest_cmd.rs +++ b/src/cmds/js/vitest_cmd.rs @@ -4,12 +4,12 @@ use anyhow::{Context, Result}; use regex::Regex; use serde::Deserialize; +use crate::core::tracking; +use crate::core::utils::{package_manager_exec, strip_ansi}; use crate::parser::{ emit_degradation_warning, emit_passthrough_warning, extract_json_object, truncate_passthrough, FormatMode, OutputParser, ParseResult, TestFailure, TestResult, TokenFormatter, }; -use crate::core::tracking; -use crate::core::utils::{package_manager_exec, strip_ansi}; /// Vitest JSON output structures (tool-specific format) #[derive(Debug, Deserialize)] diff --git a/src/cmds/ruby/rake_cmd.rs b/src/cmds/ruby/rake_cmd.rs index c0f21f61..e116c8f9 100644 --- a/src/cmds/ruby/rake_cmd.rs +++ b/src/cmds/ruby/rake_cmd.rs @@ -236,7 +236,10 @@ fn build_minitest_summary(summary: &str, failures: &[String]) -> String { for line in lines.iter().skip(1).take(4) { let trimmed = line.trim(); if !trimmed.is_empty() { - result.push_str(&format!(" {}\n", crate::core::utils::truncate(trimmed, 120))); + result.push_str(&format!( + " {}\n", + crate::core::utils::truncate(trimmed, 120) + )); } } if i < failures.len().min(10) - 1 { diff --git a/src/cmds/rust/cargo_cmd.rs b/src/cmds/rust/cargo_cmd.rs index 90416a19..60349302 100644 --- a/src/cmds/rust/cargo_cmd.rs +++ b/src/cmds/rust/cargo_cmd.rs @@ -99,7 +99,8 @@ where .unwrap_or(if output.status.success() { 0 } else { 1 }); let filtered = filter_fn(&raw); - if let Some(hint) = crate::core::tee::tee_and_hint(&raw, &format!("cargo_{}", subcommand), exit_code) + if let Some(hint) = + crate::core::tee::tee_and_hint(&raw, &format!("cargo_{}", subcommand), exit_code) { println!("{}\n{}", filtered, hint); } else { diff --git a/src/cmds/system/format_cmd.rs b/src/cmds/system/format_cmd.rs index b9f018f6..4c4a31f4 100644 --- a/src/cmds/system/format_cmd.rs +++ b/src/cmds/system/format_cmd.rs @@ -1,9 +1,9 @@ //! Runs code formatters (Prettier, Ruff) and shows only files that changed. -use crate::prettier_cmd; -use crate::ruff_cmd; use crate::core::tracking; use crate::core::utils::{package_manager_exec, resolved_command}; +use crate::prettier_cmd; +use crate::ruff_cmd; use anyhow::{Context, Result}; use std::path::Path; diff --git a/src/core/toml_filter.rs b/src/core/toml_filter.rs index b2047e61..62597717 100644 --- a/src/core/toml_filter.rs +++ b/src/core/toml_filter.rs @@ -188,7 +188,8 @@ impl TomlFilterRegistry { .unwrap_or(crate::hooks::trust::TrustStatus::Untrusted); match trust_status { - crate::hooks::trust::TrustStatus::Trusted | crate::hooks::trust::TrustStatus::EnvOverride => { + crate::hooks::trust::TrustStatus::Trusted + | crate::hooks::trust::TrustStatus::EnvOverride => { if let Ok(content) = std::fs::read_to_string(project_filter_path) { match Self::parse_and_compile(&content, "project") { Ok(f) => filters.extend(f), @@ -550,10 +551,11 @@ pub fn run_filter_tests(filter_name_opt: Option<&str>) -> VerifyResults { // Trust-gated: only verify project-local filters if trusted (SA-2025-RTK-002) let project_path = std::path::Path::new(".rtk/filters.toml"); if project_path.exists() { - let trust_status = - crate::hooks::trust::check_trust(project_path).unwrap_or(crate::hooks::trust::TrustStatus::Untrusted); + let trust_status = crate::hooks::trust::check_trust(project_path) + .unwrap_or(crate::hooks::trust::TrustStatus::Untrusted); match trust_status { - crate::hooks::trust::TrustStatus::Trusted | crate::hooks::trust::TrustStatus::EnvOverride => { + crate::hooks::trust::TrustStatus::Trusted + | crate::hooks::trust::TrustStatus::EnvOverride => { if let Ok(content) = std::fs::read_to_string(project_path) { collect_test_outcomes( &content, diff --git a/src/hooks/mod.rs b/src/hooks/mod.rs index 9c8d451e..9904d898 100644 --- a/src/hooks/mod.rs +++ b/src/hooks/mod.rs @@ -5,6 +5,7 @@ pub mod hook_check; pub mod hook_cmd; pub mod init; pub mod integrity; +pub mod permissions; pub mod rewrite_cmd; pub mod trust; pub mod verify_cmd; diff --git a/src/hooks/rewrite_cmd.rs b/src/hooks/rewrite_cmd.rs index 64bb733f..7b262be5 100644 --- a/src/hooks/rewrite_cmd.rs +++ b/src/hooks/rewrite_cmd.rs @@ -1,7 +1,7 @@ //! Translates a raw shell command into its RTK-optimized equivalent. +use super::permissions::{check_command, PermissionVerdict}; use crate::discover::registry; -use crate::permissions::{check_command, PermissionVerdict}; use std::io::Write; /// Run the `rtk rewrite` command. From 91992fd8dc4c7c87f4fd744672f9bf0909c67ba9 Mon Sep 17 00:00:00 2001 From: aesoft <43991222+aeppling@users.noreply.github.com> Date: Wed, 25 Mar 2026 20:27:53 +0100 Subject: [PATCH 5/5] fix(refacto): rm old tmp file --- TEST_EXEC_TIME.md | 102 ---------------------------------------------- 1 file changed, 102 deletions(-) delete mode 100644 TEST_EXEC_TIME.md diff --git a/TEST_EXEC_TIME.md b/TEST_EXEC_TIME.md deleted file mode 100644 index e863a268..00000000 --- a/TEST_EXEC_TIME.md +++ /dev/null @@ -1,102 +0,0 @@ -# Testing Execution Time Tracking - -## Quick Test - -```bash -# 1. Install latest version -cargo install --path . - -# 2. Run a few commands to populate data -rtk git status -rtk ls . -rtk grep "tracking" src/ - -# 3. Check gain stats (should show execution times) -rtk gain - -# Expected output: -# Total exec time: XX.Xs (avg XXms) -# By Command table should show Time column -``` - -## Detailed Test Scenarios - -### 1. Basic Time Tracking -```bash -# Run commands with different execution times -rtk git log -10 # Fast (~10ms) -rtk cargo test # Slow (~300ms) -rtk vitest run # Very slow (seconds) - -# Verify times are recorded -rtk gain -# Should show different avg times per command -``` - -### 2. Daily Breakdown -```bash -rtk gain --daily - -# Expected: -# Date column + Time column showing avg time per day -# Today should have non-zero times -# Historical data shows 0ms (no time recorded) -``` - -### 3. Export Formats - -**JSON Export:** -```bash -rtk gain --daily --format json | jq '.summary' - -# Should include: -# "total_time_ms": 12345, -# "avg_time_ms": 67 -``` - -**CSV Export:** -```bash -rtk gain --daily --format csv - -# Headers should include: -# date,commands,input_tokens,...,total_time_ms,avg_time_ms -``` - -### 4. Multiple Commands -```bash -# Run 10 commands and measure total time -for i in {1..10}; do rtk git status; done - -rtk gain -# Total exec time should be ~10-50ms (10 × 1-5ms) -``` - -## Verification Checklist - -- [ ] `rtk gain` shows "Total exec time: X (avg Yms)" -- [ ] By Command table has "Time" column -- [ ] `rtk gain --daily` shows time per day -- [ ] JSON export includes `total_time_ms` and `avg_time_ms` -- [ ] CSV export has time columns -- [ ] New commands show realistic times (not 0ms) -- [ ] Historical data preserved (old entries show 0ms) - -## Database Schema Verification - -```bash -# Check SQLite schema includes exec_time_ms -sqlite3 ~/.local/share/rtk/history.db "PRAGMA table_info(commands);" - -# Should show: -# ... -# 7|exec_time_ms|INTEGER|0|0|0 -``` - -## Performance Impact - -The timer adds negligible overhead: -- `Instant::now()` → ~10-50ns -- `elapsed()` → ~10-50ns -- SQLite insert with extra column → ~1-5µs - -Total overhead: **< 0.1ms per command**