Skip to content

Show live browser pane in agent chat view (toggle on/off) #39

@obaid

Description

@obaid

Summary

Add a "Show browser" toggle in the agent chat view (resources/js/pages/agents/chat.tsx) that splits the screen 50/50 — chat on the left, a live noVNC iframe of the agent's Chrome on the right. Toggle button in the chat header.

Why

Right now users navigate to two different routes (/agents/<id>/chat and /agents/<id>/browser) to talk to an agent and watch what it's doing. Tab-switching breaks the "watching it work" loop that builds trust during a task.

Priority / sequencing

P3 — queued behind agent stability work. The bigger lift right now is making the agent itself reliable for long-running tasks (browser-profiles wipe, gateway thrash, etc. — see #27, #28, etc.). This is a UX polish that becomes much more valuable after agents are stable, because watching an unreliable agent flail is worse than not watching at all.

Acceptance criteria

  • New "Show browser" toggle button in the chat header (similar visual weight to "More actions")
  • When toggled on: chat shrinks to ~50% width, right pane mounts an <iframe> with src="https://<server-ip>/browser/agent-<slug>/" (the existing Caddy-proxied noVNC route)
  • Toggle state persists in localStorage per agent (default: off on <lg viewport, on for lg+)
  • Iframe shows a loading spinner until VNC connects; shows a "Browser unavailable" empty state if the agent's browser service isn't running
  • Toggle gracefully collapses on mobile (<md): show a "View browser" link that opens the existing /browser tab in a new window instead of trying to split
  • The browser pane has a small "Open in new tab" link in its top-right
  • No new network calls — uses the same Caddy-proxied URL the standalone /browser page already uses

Implementation notes

  • The browser URL per agent is constructed in app/Services/AgentInstallScriptService.php:730-870 and proxied via Caddy. The existing agents/show.tsx (or wherever the Browser tab lives) already builds this URL — same logic applies.
  • VNC connection via websockify on port 6080 + display_num (per-agent display num is now persisted on agents.browser_display_num after Fix #27: re-emit browser.profiles in agent update path #32).
  • Chat container at agents/chat.tsx is currently full-width. Wrap in a flex container with the right pane conditional on toggle state.
  • Mobile breakpoint: use Tailwind's lg: for the split, otherwise show chat-only.

Out of scope

  • Interactive control of the agent's browser from this pane (clicking inside the iframe to drive the agent). The noVNC iframe is view-only / passive in this PR.
  • Cross-agent browser inbox (one place to see what all agents are doing). Separate ticket.
  • Recording / replay of the agent's session. Separate.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions