Skip to content

codex: install a standing rule block and flag untrusted hooks - #348

Merged
zzet merged 2 commits into
mainfrom
fix/codex-global-instructions-file
Jul 25, 2026
Merged

codex: install a standing rule block and flag untrusted hooks#348
zzet merged 2 commits into
mainfrom
fix/codex-global-instructions-file

Conversation

@zzet

@zzet zzet commented Jul 25, 2026

Copy link
Copy Markdown
Owner

The report

A Codex user on v0.61.2: MCP server connected, hooks present in ~/.codex/config.toml, savings dashboard at 8–25%. Adding the workflow section to AGENTS.md by hand improved things; removing it again "completely stopped index_health and smart_context from being called."

That last data point is the tell. If the hooks were working, deleting a markdown block could not take tool usage to zero — SessionStart would still be re-stating the rule every session.

Two silent failures

1. Codex skips hooks until they are trusted. Codex records trust against each non-managed hook's current hash and skips new or changed hooks until the user reviews them in /hooks. A fresh gortex install leaves SessionStart, UserPromptSubmit, PreToolUse and PostToolUse configured but never firing — and config.toml looks byte-identical whether they are trusted or skipped. Every upgrade that changes a hook definition invalidates the trust again, which is exactly the "I run gortex install after every update" workflow. We never
mentioned this anywhere.

2. Codex had no instructions surface at all. Claude Code gets ~/.claude/CLAUDE.md with an @-include of the active instruction profile. Codex got the repo-local communities routing block and nothing else. agents.InstructionsBody and agents.AppendInstructions still exist, but no adapter has called them since the install/init split, so a Codex session whose hooks were skipped carried no standing rule whatsoever and fell back to shell reads and greps.

Together: hooks inert, instructions absent, integration "active" and doing nothing.

The change

  • gortex install merges the active instruction profile into ~/.codex/AGENTS.md, between the same <!-- gortex:rules:start/end --> fence Claude Code uses. Codex loads that file into every session, so it is the Codex analogue of CLAUDE.md. The body is inlined, not @-included — Codex reads AGENTS.md as literal markdown, so a pointer line is just prose to it. gortex instructions switch refreshes the copy in place. Upsert, not append, so re-running install after every upgrade never grows the file. Honours --no-claude-md; project mode (gortex init) does not touch it. If ~/.codex/AGENTS.override.md exists, the installer says which file Codex will actually read.
  • Whenever a run writes or changes a Codex hook, the install summary carries the trust step. It stays quiet on an unchanged hook set — precisely when the recorded trust hashes still match.
  • docs/agents.md documents both, with the trust requirement called out as a blockquote in the codex section.
  • Instructions-dir resolution moves into internal/agents so the Claude and Codex writers cannot drift on where the active profile lives.

This is defence in depth, not a replacement for the hooks: SessionStart remains the strongest surface, since Codex lets AGENTS.md fade as context grows. The point is that when the hooks are skipped, the session is no
longer left with nothing — and the user is told why.

Tests

Six new cases in internal/agents/codex: content and sentinel, marker fence, inlined-not-pointer, idempotent across three applies, merges into a pre-existing personal AGENTS.md without clobbering it, --no-claude-md
and project-mode opt-outs, dry-run plans without writing, and the trust notice firing on first install but not on an unchanged re-run.

go build ./..., go vet, and go test ./internal/agents/... ./cmd/gortex/ all pass (1000 tests, 23 packages). Verified end-to-end against a throwaway $HOME: install writes the block, instructions switch localization rewrites it in place, and the block count stays at 1.

Not in this PR

  • The per-turn UserPromptSubmit hook emits nothing when the prompt probe returns no graph hits, so long Codex sessions get no re-surfacing on those turns. Worth measuring before changing — it is a per-turn token cost for every agent.
  • Savings-dashboard attribution: captureModelHint only ever writes a model for Claude Code payloads, so a Codex-only machine can never populate "cost avoided per model", and per-client call counts stall well below the recorded totals.

zzet added 2 commits July 25, 2026 23:13
A Codex install wired the MCP server and the lifecycle hooks and then
went quiet, because both of the surfaces that put the Gortex rule in
front of the model can be inert without saying so:

- Codex records trust against each non-managed hook's current hash and
  skips new or changed hooks until the user reviews them in `/hooks`.
  A fresh install therefore leaves SessionStart, UserPromptSubmit,
  PreToolUse, and PostToolUse configured but never firing, and
  config.toml looks identical in both states. Every upgrade that
  changes a hook definition re-invalidates the trust.
- Nothing was ever written to the Codex instructions surface. Claude
  Code gets ~/.claude/CLAUDE.md with an @-include of the active
  instruction profile; Codex got the repo communities block and
  nothing else, so a session whose hooks were skipped had no standing
  rule at all and fell back to shell reads and greps.

`gortex install` now merges the active instruction profile into
~/.codex/AGENTS.md between the same marker fence Claude Code uses.
Codex loads that file into every session, which makes it the Codex
analogue of CLAUDE.md. The body is inlined rather than @-included
because Codex reads AGENTS.md as literal markdown; `gortex
instructions switch` refreshes the copy in place, and the block is
upserted so re-running install after an upgrade never appends a second
one. Skipped by --no-claude-md and by project mode. When
AGENTS.override.md exists the installer says which file Codex will
actually read.

Whenever a run writes or changes a Codex hook, the install summary now
carries the trust step. It stays quiet when the hooks are unchanged,
which is exactly when the recorded trust hashes still match.

Also hoists the instructions-dir resolution into internal/agents so the
Claude and Codex writers cannot drift apart on where the active profile
lives.
The private helper had been reduced to a one-line forward, which reads
as dead weight at the definition and hides the shared resolver at the
call sites.
@zzet
zzet merged commit 95f19d1 into main Jul 25, 2026
11 checks passed
@zzet

zzet commented Jul 26, 2026

Copy link
Copy Markdown
Owner Author

Diagnostic for the reporter — scripts/gortex-codex-report.py

Pushed as a third commit. It answers "is this actually happening to me?" without a rebuild, which matters here since the reporter builds from source.

python3 scripts/gortex-codex-report.py            # last 7 days, redacted
python3 scripts/gortex-codex-report.py --days 30 --json

It joins four sources that were never correlated before:

section question it answers
Codex config are the hooks and the rule block on disk?
hook-effectiveness.jsonl did the hook processes run, inject, reach the daemon?
Codex rollouts did the model call Gortex tools, or shell out?
savings ledger what got recorded, and how much carries no client/model?

The decisive line is hooks configured, zero invocations — the untrusted-hook signature, reported as a blocker with the /hooks remedy and a non-zero exit.

Verified against a faithful reproduction of the reported state (config populated, effectiveness log empty):

✗ BLOCKER 5 Gortex hooks are configured but NONE has run in this window. Codex records
          trust against each hook's hash and skips new or changed hooks until you
          approve them. Run `/hooks` inside Codex, review the gortex entries, trust them.

And against a healthy machine, where it surfaced three things worth noting on their own:

! WARN  UserPromptSubmit ran 8 time(s) and injected context 0 times
! WARN  PostToolUse ran 60 time(s) and injected context 0 times
✓ OK    Gortex tools are 76% of tool calls (98 Gortex vs 31 shell)

That first line is the follow-up already noted in the PR body — the per-turn hook is firing and saying nothing on every turn, so long Codex sessions get no re-surfacing at all. Worth measuring separately.

Design notes for review:

  • Redacted by default (repo paths, cwds, branches → stable short hash) because the output exists to be pasted into an issue; --no-redact opts out. Prompt and source text are never read.
  • Savings DB opened immutable=1 so a live daemon is neither blocked nor disturbed.
  • The session section cannot prove a hook fired. Codex does not persist hook-injected context into rollouts (verified on 0.145.0), so it reports adoption only and says so in the output — otherwise a zero there reads as "the hook never fired", which would be wrong.

Two gaps it exposes that are not fixed here:

  1. hook-effectiveness.jsonl has no agent dimension, so on a machine running both Claude Code and Codex the counts are the sum. The hook command already knows (--agent=codex); the log just doesn't record it. Small change, real diagnostic value.
  2. Per-client attribution worked on my machine (0 unattributed events) but stalled on the reporter's — a differential worth chasing when someone repros it.

@zzet

zzet commented Jul 26, 2026

Copy link
Copy Markdown
Owner Author

Replaced the Python script with gortex doctor

Both points were right — a Python script in a Go repo is a smell, and there was prior art I should have extended instead of working around.

On gortex audit: different concept. It's a repo-level A–F code-health grade from graph topology, for the README badge. Nothing to do with installation state.

gortex init doctor was the real prior art, and it was wrong in two ways: the name reads repo-local while the report has always been machine-wide, and it only ever described configuration — the half that can't answer the question people run doctor to answer. So it's now gortex doctor, with init doctor kept as a deprecated alias.

The point: an agent config is not evidence. Codex records trust against each hook's hash and skips new or changed hooks until approved in /hooks, so a fully populated config.toml and a completely inert integration are byte-identical on disk. The old doctor would have called the reporter's machine healthy. The distinguishing signal already existed and was never read — every hook process appends a row to hook-effectiveness.jsonl whether or not it injected anything, so an event with zero rows did not run.

Three new sections beside the adapter drift it always showed:

section question
hook activity runs / injections / daemon reachability per event
adoption gortex tool calls vs shell calls, from the agent's transcripts
savings what the ledger recorded, how much has no client or model

On the reproduced report state:

✗ BLOCKER 5 codex hook(s) are configured but none has run in this window —
          codex skips new or changed hooks until they are trusted.
          → run `/hooks` inside Codex, review the gortex entries, and trust them

Exit status is non-zero on a blocker so CI can gate; --redact hashes repo paths and branches for pasting into an issue; --json for machines.

Structure

  • internal/hooks/telemetry_read.go — reader next to the writer, so the record shape has one definition.
  • internal/doctor — transcript scanning and the findings table, as pure data in / findings out. The whole decision table is testable without a machine that has Codex, hooks, or a daemon on it.
  • codex.Inspect — real TOML parsing that reuses the adapter's own hook recognisers, with a test that installs and then inspects. If Apply's hook shape ever changes without Inspect following, that test fails instead of doctor quietly reporting a correct install as unconfigured.

One deliberate omission

The adoption section carries no "did the hook fire" signal. I'd planned to check for the SessionStart orientation block in transcripts — then verified Codex 0.145.0 doesn't persist hook-injected context into rollouts at all. An absent block proves nothing, so the section says so in its own output rather than inviting a zero to be misread.

Tests

37 new cases across the three packages: window filtering and torn-line tolerance in the log reader; tool classification, window/empty-session exclusion, and garbage tolerance in the transcript scanner; and the full findings table — never-ran vs ran-but-silent vs daemon-unreachable, trust wording confined to trust-gating agents, shadowed-vs-missing instructions not double-reported, blockers sorting first.

go build ./..., go vet ./..., go test ./internal/... ./cmd/gortex/ — 12,012 tests across 169 packages, green.

@zzet

zzet commented Jul 26, 2026

Copy link
Copy Markdown
Owner Author

Split the diagnostic work out into #351 (based on this branch), so this PR is back to the two commits it should be: the ~/.codex/AGENTS.md rule block and the hook-trust notice.

Force-pushed to drop the two commits that moved — the Python script I'd added and its Go replacement. Nothing was reviewed at those SHAs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant