Repository navigation
feat: let agents drive a browser in the sandbox, and people use it with them - #303
Merged
Merged
Conversation
tjholm
added this pull request to stack #305
October 6, 2026 03:35
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
sugabots-website | 6e9e8ea | Oct 07 2026, 11:44 PM |
tjholm
force-pushed
the
sandbox-desktop
branch
from
October 6, 2026 05:09
7b9a898 to
a2c3912
Compare
tjholm
force-pushed
the
sandbox-desktop
branch
from
October 6, 2026 21:29
a2c3912 to
94591c9
Compare
tjholm
force-pushed
the
sandbox-desktop
branch
from
October 7, 2026 02:48
94591c9 to
1ba02c4
Compare
tjholm
marked this pull request as ready for review
October 7, 2026 02:56
tjholm
removed this pull request from stack #305
October 7, 2026 02:58
tjholm
added this pull request to stack #327
October 7, 2026 02:58
tjholm
force-pushed
the
sandbox-desktop
branch
from
October 7, 2026 03:58
1ba02c4 to
6ec7359
Compare
tjholm
force-pushed
the
sandbox-desktop
branch
3 times, most recently
from
October 7, 2026 06:27
f52f948 to
1ea641d
Compare
tjholm
force-pushed
the
sandbox-desktop
branch
from
October 7, 2026 07:35
1ea641d to
3478222
Compare
tjholm
force-pushed
the
sandbox-desktop
branch
from
October 7, 2026 07:38
3478222 to
f05983f
Compare
tjholm
force-pushed
the
sandbox-desktop
branch
from
October 7, 2026 08:20
f05983f to
629f4d6
Compare
|
Sandbox image preview, built from 6e9e8ea: To try it, set a sandbox provider's image to that under Sandboxes in the workspace's settings, then Upgrade the pod's sandbox; on E2B, prepare the template again first. The tag moves with each push to this pull request, so a machine that pulled it before may need It's deleted when the pull request closes. |
tjholm
force-pushed
the
sandbox-desktop
branch
4 times, most recently
from
October 7, 2026 22:39
d6b68c4 to
3e32870
Compare
tjholm
force-pushed
the
sandbox-desktop
branch
3 times, most recently
from
October 7, 2026 23:29
c662645 to
400b52b
Compare
…th them Agents get Playwright MCP's browser_ tools, run by a Chromium of their own in each thread on their own desktop in the pod's sandbox. Chromium keeps its own sandbox where the sandbox allows user namespaces, and runs without one where it doesn't. A managed policy keeps pages in it from reaching the sandbox's own network, where Playwright MCP listens without a password, and has it connect without QUIC or Encrypted Client Hello, so egress rules that allow hosts by the name a TLS connection gives let it through. Each desktop's VNC servers take passwords made when it starts, which the API answers with on the person's behalf, so neither the passwords nor the sandbox's address reach the browser. People open that desktop from the Sandbox button at the top of the agent's chat, which says when the agent is using it, or from a browser call in the thread, through a WebSocket the API relays from the desktop's VNC server. Anyone who can read the thread may open the desktop of an agent that takes part in it and uses the sandbox; those with the new sandbox.desktop.use pod permission (members and up) may click and type, and the pod's viewers watch through a view-only VNC server. The viewer disconnects after 5 minutes without input from the person or browser use by the agent. Opening it uses the sandbox as a turn would, resuming or making it, holds it awake, and starts the desktop without a browser; the agent's browser joins the same desktop. read_file shows images to models that take them. The sandbox interface gains endpoint(port), through OpenSandbox's port proxy or E2B's host, with header routing on E2B Embed.
tjholm
force-pushed
the
sandbox-desktop
branch
from
October 7, 2026 23:43
400b52b to
6e9e8ea
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Agent browser
browser_*tools.Watching and taking over
Other
read_fileshows images to models that accept them.endpoint(port): OpenSandbox's port proxy, or E2B's host (header routing on E2B Embed).Stack created with GitHub Stacks CLI • Give Feedback 💬