Skip to content

Commit e7c057c

Browse files
wan9chiwan9chi
authored
test(e2e): cover fspy in agent sandboxes (#564)
## Motivation `vt` is commonly run from Codex CLI and Claude Code sandboxed shells. Add macOS e2e coverage that invokes `vt` directly inside `@openai/codex` and `@anthropic-ai/sandbox-runtime` using profiles that reflect the harness defaults. The snapshots intentionally preserve the current runner IPC failures; this PR only records the behavior and does not fix it. Refs #561, #562, and #563.
1 parent ea5ce86 commit e7c057c

10 files changed

Lines changed: 292 additions & 0 deletions

File tree

‎crates/vite_task_bin/tests/e2e_snapshots/fixtures/sandboxed_fspy/claude-code-default-sandbox.json‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
{
2+
"network": {
3+
"allowedDomains": [],
4+
"deniedDomains": []
5+
},
6+
"filesystem": {
7+
"denyRead": [],
8+
"allowRead": [],
9+
"allowWrite": ["."],
10+
"denyWrite": []
11+
}
12+
}

‎crates/vite_task_bin/tests/e2e_snapshots/fixtures/sandboxed_fspy/input.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
tracked input

‎crates/vite_task_bin/tests/e2e_snapshots/fixtures/sandboxed_fspy/package.json‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{
2+
"name": "sandboxed-fspy",
3+
"private": true
4+
}

‎crates/vite_task_bin/tests/e2e_snapshots/fixtures/sandboxed_fspy/snapshots.toml‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
[[e2e]]
2+
name = "fspy_under_codex_sandbox"
3+
comment = """
4+
Runs `vt run inner` under Codex CLI's built-in `:workspace` permission profile, representing its default workspace-editing posture for a trusted repository. The profile makes the workspace roots and system temp directories writable without adding network or Unix socket access. See [Codex permissions](https://learn.chatgpt.com/docs/permissions#define-and-select-a-profile).
5+
6+
The nested `vt` enables fspy for automatic input inference; changing the file read inside the sandbox checks whether it invalidates the cache.
7+
"""
8+
platform = "macos"
9+
ignore = true
10+
steps = [
11+
[
12+
"codex",
13+
"sandbox",
14+
"-P",
15+
":workspace",
16+
"vt",
17+
"run",
18+
"inner",
19+
],
20+
[
21+
"vtt",
22+
"replace-file-content",
23+
"input.txt",
24+
"tracked",
25+
"modified",
26+
],
27+
[
28+
"codex",
29+
"sandbox",
30+
"-P",
31+
":workspace",
32+
"vt",
33+
"run",
34+
"inner",
35+
],
36+
]
37+
38+
[[e2e]]
39+
name = "fspy_under_anthropic_sandbox_runtime"
40+
comment = """
41+
Runs `vt run inner` under a profile matching Claude Code's default enabled Bash sandbox: the working directory and session temp directory are writable, with no pre-allowed network domains or extra Unix socket access. See [Claude Code sandboxing](https://code.claude.com/docs/en/sandboxing#filesystem-isolation).
42+
43+
Claude Code normally creates the writable session temp before invoking Sandbox Runtime, so the harness creates the runtime's default `/tmp/claude` directory before calling `srt` directly. The fixture profile provides only `allowWrite: ["."]`; Sandbox Runtime supplies `/tmp/claude` and its other built-in compatibility paths. See [`getDefaultWritePaths` and `generateProxyEnvVars`](https://github.com/anthropic-experimental/sandbox-runtime/blob/main/src/sandbox/sandbox-utils.ts).
44+
45+
The nested `vt` enables fspy for automatic input inference; changing the file read inside the sandbox checks whether it invalidates the cache.
46+
"""
47+
platform = "macos"
48+
ignore = true
49+
steps = [
50+
{ argv = [
51+
"mkdir",
52+
"-p",
53+
"/tmp/claude",
54+
], comment = "create the session temp that Claude Code supplies before sandboxed Bash commands" },
55+
[
56+
"srt",
57+
"--settings",
58+
"claude-code-default-sandbox.json",
59+
"vt",
60+
"run",
61+
"inner",
62+
],
63+
[
64+
"vtt",
65+
"replace-file-content",
66+
"input.txt",
67+
"tracked",
68+
"modified",
69+
],
70+
[
71+
"srt",
72+
"--settings",
73+
"claude-code-default-sandbox.json",
74+
"vt",
75+
"run",
76+
"inner",
77+
],
78+
]

‎crates/vite_task_bin/tests/e2e_snapshots/fixtures/sandboxed_fspy/snapshots/fspy_under_anthropic_sandbox_runtime.md‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# fspy_under_anthropic_sandbox_runtime
2+
3+
Runs `vt run inner` under a profile matching Claude Code's default enabled Bash sandbox: the working directory and session temp directory are writable, with no pre-allowed network domains or extra Unix socket access. See [Claude Code sandboxing](https://code.claude.com/docs/en/sandboxing#filesystem-isolation).
4+
5+
Claude Code normally creates the writable session temp before invoking Sandbox Runtime, so the harness creates the runtime's default `/tmp/claude` directory before calling `srt` directly. The fixture profile provides only `allowWrite: ["."]`; Sandbox Runtime supplies `/tmp/claude` and its other built-in compatibility paths. See [`getDefaultWritePaths` and `generateProxyEnvVars`](https://github.com/anthropic-experimental/sandbox-runtime/blob/main/src/sandbox/sandbox-utils.ts).
6+
7+
The nested `vt` enables fspy for automatic input inference; changing the file read inside the sandbox checks whether it invalidates the cache.
8+
9+
## `mkdir -p /tmp/claude`
10+
11+
create the session temp that Claude Code supplies before sandboxed Bash commands
12+
13+
```
14+
```
15+
16+
## `srt --settings claude-code-default-sandbox.json vt run inner`
17+
18+
**Exit code:** 1
19+
20+
```
21+
$ vtt print-file input.txt
22+
✗ Failed to set up task communication: Operation not permitted (os error 1)
23+
```
24+
25+
## `vtt replace-file-content input.txt tracked modified`
26+
27+
```
28+
```
29+
30+
## `srt --settings claude-code-default-sandbox.json vt run inner`
31+
32+
**Exit code:** 1
33+
34+
```
35+
$ vtt print-file input.txt
36+
✗ Failed to set up task communication: Operation not permitted (os error 1)
37+
```

‎crates/vite_task_bin/tests/e2e_snapshots/fixtures/sandboxed_fspy/snapshots/fspy_under_codex_sandbox.md‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
# fspy_under_codex_sandbox
2+
3+
Runs `vt run inner` under Codex CLI's built-in `:workspace` permission profile, representing its default workspace-editing posture for a trusted repository. The profile makes the workspace roots and system temp directories writable without adding network or Unix socket access. See [Codex permissions](https://learn.chatgpt.com/docs/permissions#define-and-select-a-profile).
4+
5+
The nested `vt` enables fspy for automatic input inference; changing the file read inside the sandbox checks whether it invalidates the cache.
6+
7+
## `codex sandbox -P :workspace vt run inner`
8+
9+
**Exit code:** 1
10+
11+
```
12+
$ vtt print-file input.txt
13+
✗ Failed to set up task communication: Operation not permitted (os error 1)
14+
```
15+
16+
## `vtt replace-file-content input.txt tracked modified`
17+
18+
```
19+
```
20+
21+
## `codex sandbox -P :workspace vt run inner`
22+
23+
**Exit code:** 1
24+
25+
```
26+
$ vtt print-file input.txt
27+
✗ Failed to set up task communication: Operation not permitted (os error 1)
28+
```

‎crates/vite_task_bin/tests/e2e_snapshots/fixtures/sandboxed_fspy/vite-task.json‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
{
2+
"tasks": {
3+
"inner": {
4+
"command": "vtt print-file input.txt",
5+
"cache": true,
6+
"input": [
7+
{
8+
"auto": true
9+
}
10+
],
11+
"output": []
12+
}
13+
}
14+
}

‎packages/tools/package.json‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,8 @@
33
"private": true,
44
"type": "module",
55
"dependencies": {
6+
"@anthropic-ai/sandbox-runtime": "catalog:",
7+
"@openai/codex": "catalog:",
68
"@playwright/browser-chromium": "catalog:",
79
"@vitest/browser-playwright": "catalog:",
810
"@voidzero-dev/vite-task-client": "workspace:*",

‎pnpm-lock.yaml‎

Lines changed: 114 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)