RFC: Graduating Collie into a standalone mobile agent multiplexer

[AltanS]

RFC: Graduating Collie into a standalone mobile agent multiplexer

Status: exploratory. Nothing here is committed. This thread is the place to flesh the idea out. If you run Collie, or would if it didn't require Herdr, your input shapes whether and how this happens.

The idea

Collie grows from a Herdr plugin into a self-hosted mobile control plane for a herd of terminal coding agents, attaching to sessions you already run under any supported multiplexer (Herdr, tmux, ...), never owning the agent processes.

What it would give us

• Bring your own multiplexer. Run your herd under tmux or Herdr and monitor, triage and reply from the same phone UI. New backends are drivers, not rewrites.
• Mixed herds. Claude, Codex, pi and opencode side by side in one view, regardless of where each one runs.
• Any agent works on day one. The baseline, a screen mirror plus guarded key injection, needs no agent-specific integration. Brand-new CLIs are usable immediately, harness adapters upgrade them over time.
• Deeper integration where a harness allows it. Where an agent exposes a structured surface (hooks, session servers), Collie answers permission prompts and dialogs as typed contracts instead of synthesized keystrokes, and gets reliable state for notifications.
• Capabilities the current setup can't offer: real scrollback, genuine bracketed paste, headless control over pane geometry.
• Existing setups keep working. Herdr becomes the first driver, and current installs continue through a thin plugin shim.

Architecture, high level

Two adapter axes plus a coordination layer:

flowchart TD
    pwa["Phone PWA"] <-->|"blocks, the stable UI contract"| shepherd
    shepherd["<b>Shepherd</b><br/>session registry, status ladder, triage"]
    shepherd <--> harness
    shepherd <--> mux
    harness["<b>Harness surfaces</b> (semantics)<br/>claude, codex, pi, opencode, ..."]
    mux["<b>Mux drivers</b> (transport)<br/>herdr, tmux, ..."]
    harness -.->|"hooks, session APIs"| sessions
    mux <-->|"rendered grid, keys"| sessions
    sessions["Agent sessions in your terminals"]

Loading

• The session is the subject, not the pane. A pane is one way to present a session. Identity, drafts and triage key on sessions, and the mux is how sessions are found and rendered.
• Mux drivers answer "what's running, what does it look like, how do I type": topology, rendered-grid reads, key injection through one canonical key model, change signals, and a capability declaration the UI feature-gates on. Drivers must deliver a rendered grid, Collie still never runs a terminal emulator (ADR 0008 becomes a driver requirement).
• Harness surfaces answer "what does this screen mean, and how do I answer safely". Each harness sits on a capability ladder: a native session server where one exists, grid plus semantic side channels (hooks) where available, plain grid as the universal floor.
• The shepherd provides what a generic multiplexer doesn't: which panes are agents, which harness they run, and whether they need attention. Status comes from the best available source (driver, harness hooks, screen heuristics), and recovery controls like interrupt never depend on inferred state.

History stays the journal's job: the screen is now, the journal is the past.

What stays true

• No terminal emulator in Collie, ever.
• One bridge per host, one managed front door (ADR 0001). The session registry doubles as the security boundary, only registered agent panes are reachable.
• Refuse, don't guess, on unrecognized screens. The send guard stays between text and Enter.
• SemVer, typed dialog contracts, conformance tests.

Open questions

1. Would you use this? Under which multiplexer, and what does your herd look like?
2. Dual control, a phone and a desk driving one session: what guarantees would you expect?
3. Session registration: explicit tagging with heuristic discovery as convenience, or too much ceremony?
4. Status without Herdr: are hook-based state plus screen heuristics trustworthy enough for push notifications?
5. Multi-host: one bridge per host stays law. Is aggregation across hosts a real need, or YAGNI?

Nothing above is a promise, it's a direction worth pressure-testing in the open.

[bobleer]

I'd keep registration explicit and use heuristics only to suggest candidates. A missed pane is annoying; a false positive means the phone can type into the wrong shell, which is much worse. In BitFun's self-hosted relay we keep session identity separate from pane/process identity for the same reason.