Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ supported integrations:
| --- | --- | --- | --- |
| <img width="48px" src="docs/client-claude.jpg" alt="Claude" /> | [Claude Code](https://docs.anthropic.com/en/docs/claude-code) | `tokenjuice install claude-code` | `~/.claude/settings.json` |
| <img width="48px" src="docs/client-codebuddy.png" alt="CodeBuddy" /> | [CodeBuddy](https://codebuddy.tencent.com/) | `tokenjuice install codebuddy` | `~/.codebuddy/settings.json` |
| <img width="48px" src="docs/client-openai.jpg" alt="Codex" /> | [Codex CLI](https://github.com/openai/codex) | `tokenjuice install codex` | `~/.codex/hooks.json` |
| <img width="48px" src="docs/client-openai.jpg" alt="Codex" /> | [Codex CLI](https://github.com/openai/codex) ([integration notes](docs/codex-integration.md)) | `tokenjuice install codex` | `~/.codex/hooks.json` |
| <img width="48px" src="docs/client-cursor.jpg" alt="Cursor" /> | [Cursor](https://cursor.com/docs/hooks) | `tokenjuice install cursor` | `~/.cursor/hooks.json` |
| <img width="48px" src="docs/client-droid.png" alt="Droid" /> | [Droid (Factory CLI)](https://docs.factory.ai/cli/configuration/hooks-guide) | `tokenjuice install droid` | `~/.factory/settings.json` |
| <img width="48px" src="docs/client-copilot.png" alt="GitHub Copilot CLI" /> | [GitHub Copilot CLI](https://github.com/github/copilot-cli) | `tokenjuice install copilot-cli` | `~/.copilot/hooks/tokenjuice-cli.json` |
Expand Down Expand Up @@ -253,6 +253,7 @@ direct payload:
- [Coder Agents integration](docs/coder-agents-integration.md)
- [CodeRabbit integration](docs/coderabbit-integration.md)
- [Command Code integration](docs/command-code-integration.md)
- [Codex CLI integration](docs/codex-integration.md)
- [Crush integration](docs/crush-integration.md)
- [Cursor integration](docs/cursor-integration.md)
- [CodeBuddy integration](docs/codebuddy-integration.md)
Expand Down
61 changes: 61 additions & 0 deletions docs/codex-integration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Codex CLI integration

`tokenjuice install codex` adds a `PostToolUse` hook for Bash results to
`~/.codex/hooks.json`. Outputs that produce a worthwhile reduction are replaced
with compacted context; small, low-savings, and protected inspection results are
left unchanged.

## Expected replacement status

With the currently tested Codex CLI, a PostToolUse hook must return
`continue:false` to suppress the original tool result. A successful Tokenjuice
rewrite has been observed as:

```text
PostToolUse hook (stopped)
hook context: <compacted output>
stop: Tokenjuice replaced the original Bash output with the compacted context above.
```

Interpret this status as a successful replacement only when both the Tokenjuice
replacement reason and compacted hook context are present. In that case,
`stopped` describes the hook suppressing the original result; it is not the Bash
exit status. The authenticated regression verifies that the agent can produce
a later assistant response after the replacement.

Without `continue:false`, the currently tested Codex CLI retains the full
original result and adds the summary beside it, which does not provide the
intended context-token savings. Until Codex exposes a clean "replace output"
primitive, the `stopped` label is an expected UI tradeoff for real output
replacement.

Use `tokenjuice wrap --raw -- <command>` when the full command output is
required. This escape hatch reruns the command; review side effects before using
it with commands that mutate files or external systems.

## Local verification

To point the real Codex home at the current checkout:

```bash
pnpm build
node dist/cli/main.js install codex --local
node dist/cli/main.js doctor codex --local
```

`doctor` should report `status: ok`. Use it first when the hook is disabled,
stale, or missing.

Run the authenticated live regression separately:

```bash
pnpm e2e:codex-live
```

The E2E builds and installs into an isolated temporary `CODEX_HOME`; it does not
require the real-home local install above. It consumes Codex quota and requires
an existing login at `$CODEX_HOME/auth.json` (or `~/.codex/auth.json`). Set
`TOKENJUICE_CODEX_LIVE_SOURCE_HOME` to select a different authenticated home.
It verifies that the compacted context and replacement reason are model-visible,
a later assistant response is present, and the original marker is absent from
the compacted context and function-call output.
14 changes: 14 additions & 0 deletions docs/integration-playbook.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,20 @@ for truncation-related debugging, verify both boundaries explicitly:
- capture truncation: if output includes `[tokenjuice: output truncated]`, rerun with a larger capture ceiling (for example `--max-capture-bytes 52428800`).
- do not treat these as the same failure mode; reducer bypass and capture-size tuning solve different problems.

### Codex live context replacement

Run the authenticated Codex regression manually:

```bash
pnpm e2e:codex-live
```

This command builds the current source, installs its hook into an isolated temporary `CODEX_HOME`, runs a fixed fake `gh` command through the real `codex exec`, and inspects the saved transcript plus Tokenjuice hook debug output. It requires an existing Codex login at `$CODEX_HOME/auth.json` (or `~/.codex/auth.json`); set `TOKENJUICE_CODEX_LIVE_SOURCE_HOME` to select a different authenticated home.

The transcript must contain the compacted developer context, the model-visible replacement reason, and a later assistant response, while excluding the raw marker from the compacted context and function-call output. With the currently tested Codex CLI, replacement renders as `PostToolUse hook (stopped)` because `continue:false` is required to suppress the original result. Treat that status as expected only when the Tokenjuice replacement reason and compacted context are present; it describes hook replacement, not the Bash exit status.

The command invokes a model and consumes account quota, so keep it out of normal CI. It removes the temporary Codex home and authentication symlink before reporting success.

## docs updates required in same PR

when adding a host integration, update:
Expand Down
2 changes: 1 addition & 1 deletion docs/spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -380,7 +380,7 @@ supported host hooks:
| CodeRabbit | `tokenjuice install coderabbit` | `.coderabbit.yaml` | ✴️ Beta. Inserts marker-delimited `reviews.path_instructions` guidance that tells CodeRabbit review, finishing-touch, chat, and tool workflows to use `tokenjuice wrap` for noisy terminal commands and `tokenjuice wrap --raw -- <command>` only when raw bytes are needed; guidance-only, because CodeRabbit config does not intercept review comments or command output; see `docs/coderabbit-integration.md` |
| CodeBuddy (Linux/macOS/WSL) | `tokenjuice install codebuddy` | `~/.codebuddy/settings.json` | Uses `PreToolUse` shell input rewriting (same pattern as Cursor) to route Bash commands through `tokenjuice wrap`; preserves unrelated hooks that share a matcher group with the tokenjuice entry; `tokenjuice install codebuddy --local` is available for repo-local verification; native Windows shell interception is intentionally blocked for now; see `docs/codebuddy-integration.md` |
| Command Code | `tokenjuice install command-code` | `~/.commandcode/settings.json` / `.commandcode/settings.json` | ✴️ Beta. Uses a `PostToolUse` hook with matcher `shell`; compacted context is injected through `hookSpecificOutput.additionalContext` alongside the original shell output; `tokenjuice install command-code --local` is available for repo-local verification; see `docs/command-code-integration.md` |
| Codex CLI | `tokenjuice install codex` | `~/.codex/hooks.json` | `tokenjuice install codex --local` is available for repo-local verification |
| Codex CLI | `tokenjuice install codex` | `~/.codex/hooks.json` | Rewrites suppress the original Bash result and inject compacted context. With the currently tested Codex CLI, this replacement renders as `PostToolUse hook (stopped)` because output suppression requires `continue:false`; when the Tokenjuice replacement reason and compacted context are present, this is a hook-replacement status rather than a Bash failure. `tokenjuice install codex --local` is available for repo-local verification; see `docs/codex-integration.md` |
| Continue | `tokenjuice install continue` | `.continue/rules/tokenjuice.md` | ✴️ Beta. Installs a workspace rule that tells Continue agents to use `tokenjuice wrap` for noisy terminal commands and `tokenjuice wrap --raw -- <command>` only when raw bytes are needed; guidance-only, because Continue rules do not intercept tool output; see `docs/continue-integration.md` |
| Crush | `tokenjuice install crush` | `.crush/skills/tokenjuice/SKILL.md` | ✴️ Beta. Installs a project Agent Skill that tells Crush to use `tokenjuice wrap` for noisy terminal commands and `tokenjuice wrap --raw -- <command>` only when raw bytes are needed; guidance-only, because Crush hook composition and stateful shell behavior make command rewriting unsafe; see `docs/crush-integration.md` |
| Cursor (Linux/macOS/WSL) | `tokenjuice install cursor` | `~/.cursor/hooks.json` | Uses `preToolUse` shell input rewriting to route commands through `tokenjuice wrap`; `tokenjuice install cursor --local` is available for repo-local verification; native Windows shell interception is intentionally blocked for now; see `docs/cursor-integration.md` |
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@
"bench:fixtures": "node scripts/bench.mjs fixtures",
"bench:verify": "node scripts/bench.mjs verify",
"contracts": "publint run --strict --pack pnpm",
"e2e:codex-live": "pnpm build && node scripts/codex-live-e2e.mjs",
"e2e:local": "pnpm build && node scripts/local-host-e2e.mjs",
"generate:builtin-rules": "node scripts/generate-builtin-rules.mjs",
"prepare": "pnpm repair:pnpm-bin-shims",
Expand Down
Loading