Skip to content

fix(rsi): agent never calls rsi_propose, so capability-gap opportunities are answered with brain-file rules #842

Description

@adolfousier

The Mission Control inbox has been empty since the proposal stores were cleared, despite the RSI loop generating capability-gap opportunities on nearly every cycle.

Evidence

A single cycle found 17 opportunities, several naming the tool and the kind outright:

RSI cycle: analyzing 109586 feedback entries
RSI opportunity: Bash subsystem <name> has 288 successful invocations in the window.
  Promotion candidate: file a tool (rsi_propose kind=tool) ...
RSI opportunity: Tool sequence <a -> b -> c> ran in 6 distinct sessions
  - candidate for a skill (rsi_propose kind=skill) ...
RSI cycle: 17 opportunities found, spawning autonomous agent

The agent then spent 156197 tokens ($0.2343) and completed with:

RSI agent completed: Checked SOUL.md, AGENTS.md, and TOOLS.md with self_improve
reads because Bash grep is not exposed in this RSI session.

rsi_propose is registered in build_rsi_tool_registry (src/brain/rsi.rs:305). Across 8 days of logs covering 116 opportunity-bearing cycles there is not one invocation: every match is either the registration line, the opportunity text (which contains the literal tool name), or the agent quoting the name in its own streamed prose.

Ruled out as causes:

  • Empty stores. proposals = [] is the post-clear state, not a suppressor.
  • pruned.toml. That records pruned brain-file sections and has no bearing on proposals.
  • Tool availability. The tool is registered and reachable.

Diagnosis

The agent treats every opportunity as a guidance problem and answers it with self_improve, writing another SOUL.md rule. A recurring 288-invocation command shape is a capability gap: no rule can close it, so the rule is pure prompt bloat and the gap persists into the next cycle, where it is re-detected and re-answered the same way.

The prompt already states the distinction (rsi.rs:330):

rsi_propose asks for something the agent CANNOT currently do: a new tool, slash command, ...

Stating it once in a long prompt is evidently not enough to redirect the behaviour, especially when the agent is mid-way through an analysis loop that has self_improve as its habitual next step. This is the capability-versus-guidance failure the BinEval work describes, and it also matches the known prompt-bloat degradation: each unnecessary rule makes the next cycle worse.

Fix direction

  • Make the opportunity carry its own required disposition rather than leaving classification to the agent. Opportunities emitted by the promotion, command-pattern and skill-sequence detectors already know their kind; they should be non-negotiable about which tool answers them.
  • Reject the wrong answer instead of accepting it silently. A self_improve call responding to a capability-gap opportunity should fail with the reason, so the agent retries with rsi_propose rather than closing the cycle satisfied.
  • Log the disposition per cycle so a silent regression is visible: opportunities found, by kind, versus proposals actually filed.

Impact

  • Capability gaps are never surfaced to the user; the inbox stays empty by construction
  • Each cycle spends real tokens re-detecting the same gaps and answering them with rules
  • SOUL.md accumulates guidance for problems guidance cannot solve

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions