Skip to content

Commit 855acd1

Browse files
committed
test: verify Express pause and snapshot state continuity
1 parent 0d9da63 commit 855acd1

9 files changed

Lines changed: 379 additions & 6 deletions

File tree

‎.agents/README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -89,3 +89,5 @@ Builtin transport: [direct HTTP](notes/implemented/architecture/2026-10-18-nativ
8989
- [Provider-owned metadata follow-up](evidence/verify/2026-10-18-provider-owned-metadata-followup.md)
9090

9191
- [Provider preflight](notes/proposed/feature/2026-10-18-provider-preflight.md) · [verification](evidence/verify/2026-10-18-provider-preflight.md)
92+
93+
- [Conditional Express acceptance](notes/proposed/feature/2026-10-19-conditional-acceptance.md) · [verification](evidence/verify/2026-10-19-conditional-acceptance.md)
Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# Verification: conditional Express acceptance
2+
3+
- Date:2026-10-19
4+
- Result:PASS — real local acceptance; exact-head hosted CI follows push
5+
- Backend: actual Alibaba Express, isolated generation2 fixture
6+
- Credentials: local ignored config only; never uploaded or printed
7+
8+
## Design and provisioning
9+
10+
Read official generation2/snapshot contracts before changes. Read-only POP inventory
11+
found two existing ready generation2 templates. Actual built CLI submitted a new
12+
unique same-region official runtime-v2/base:v0.0.53 template with generation2,
13+
CPU2, memory2048MB, disk15360MB and explicit Team; build reached ready.
14+
Existing templates/account policy were not changed. Thus the earlier absence of
15+
conditional verification was incomplete investigation, not missing local API keys.
16+
17+
## Prior failures retained
18+
19+
Initial new pause test incorrectly passed strings instead of SandboxState enum;
20+
fixed the test input. The next run exposed409 deletion during lifecycle transition,
21+
brief snapshot-index lag, and repeated snapshot DELETE returning204/True. A scoped
22+
real diagnostic confirmed list visibility after3 seconds. Tests now wait only for
23+
known transition conflicts/index consistency, verify actual resource absence and
24+
preserve arbitrary backend errors. No client/backend success was fabricated.
25+
26+
## SDK results
27+
28+
Fresh installed paired client/server, Python3.13.13.
29+
python -m pytest test_standard_contract.py -q -ra --tb=short --capture=tee-sys:
30+
5 PASS in36.58s, zero skipped. Includes baseline sync/async/files/process/
31+
interpreter/network, actual pause-state/connect, in-memory nonce and files,
32+
snapshot create/list/restore, independent clone state and deletion. Six sandboxes
33+
tracked; cleanup zero residuals. New tests keep independent selectors and scoped
34+
reconciliation for an ambiguous snapshot timeout.
35+
36+
Maintained lint/format/typecheck PASS (server56/client105 source files); complete
37+
client2190 PASS/23 SKIP/10 integration deselected in122.19s in this all-extras
38+
development environment. No source-server implementation changed; its complete
39+
remote Python matrix is required on the new PR head.
40+
41+
## Consumer and cleanup dependencies
42+
43+
Consumers now use native api.sandbox imports that were actually checked against
44+
accepted SDK main81cca585 and standard candidate0d9da632. Review owner PRs:
45+
https://github.com/Alibaba-Agent-Sandbox/awesome-providers/pull/13
46+
https://github.com/Alibaba-Agent-Sandbox/awesome-templates/pull/115
47+
https://github.com/Alibaba-Agent-Sandbox/blogs/pull/4
48+
https://github.com/Alibaba-Agent-Sandbox/website/pull/11
49+
Registry aggregates those sources; UI/catalog contracts did not change.
50+
51+
Only this run's sandbox/snapshot/template resources are eligible for deletion.
52+
The owned fixture is removed after both artifacts finish, then independently
53+
queried for absence. Hosted results and actual merge state are separate from
54+
this local receipt. Earlier failures remain point-in-time history.
55+
56+
## Final independent cleanup
57+
58+
Actual standard CLI template delete --yes removed the unique owned fixture.
59+
Independent API GET returned404; paginated queries across all3 test run tags found
60+
zero owned sandboxes, and the snapshot list was empty. No existing template was
61+
modified or removed. An initial cleanup invocation incorrectly included native
62+
--json on the standard parser and was rejected before mutation; correct standard
63+
syntax then succeeded.
Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
# Decision: Provision conditional Express acceptance fixtures
2+
3+
Status: proposed
4+
5+
## Problem
6+
Conditional acceptance previously stopped after failures on a generation1 template;
7+
consumer documentation also depends on the old asynchronous root import/CLI flags.
8+
A valid local runtime key alone cannot select the generation or enable an allowlist.
9+
10+
## Decision
11+
Add independently selectable E2E_PAUSE_TEMPLATE and E2E_SNAPSHOT_TEMPLATE real tests.
12+
Assert running→paused→connected-running and in-memory nonce/file preservation;
13+
assert snapshot creation/list/restore/clone isolation/deletion with owned cleanup.
14+
Missing prerequisites skip only their own scenario and remain acceptance gaps.
15+
Provision only a uniquely named task fixture; inspect build state and preserve
16+
server admission errors. Do not modify existing templates, account policy or feature
17+
flags. Use local ignored credentials; keep resource identifiers out of evidence.
18+
Consumers use the stable native api.sandbox import for provider/server examples
19+
and choose CLI syntax from installed help, preserving working main installations.
20+
21+
## API Design
22+
No public client API changes. Existing OpenAPI CreateTemplate uses sandboxConfig
23+
generation2, official same-region runtime-v2/base image, explicit15360MB disk.
24+
Keep native ABX settings and standard-entry E2B mapping separate.
25+
26+
## Alternatives considered
27+
Repeating generation1 probes cannot test MicroVM behavior. Silently skipping all
28+
conditional tests or changing a gate to pass would hide missing acceptance.
29+
Changing existing cloud templates risks other sessions; use an isolated fixture.
30+
31+
## Dependencies
32+
https://github.com/Alibaba-Agent-Sandbox/python-sdk/pull/47
33+
https://github.com/Alibaba-Agent-Sandbox/agent-sandbox/pull/40
34+
Official contracts: https://help.aliyun.com/zh/agent-sandbox/build-and-manage-custom-templates-with-code
35+
and https://www.alibabacloud.com/help/en/agent-sandbox/e2b-compatibility-explanation
36+
37+
## Test Strategy
38+
Run required owner checks; actual local provisioning and cloud probes with exact
39+
source/artifact receipts; inspect owned resources after cleanup. Record failures,
40+
skips and platform allowlist restrictions separately from passes. Check consumer
41+
imports against both accepted-main and standard-candidate installations.
42+
43+
## Acceptance criteria
44+
Runnable bilingual guide and scoped consumer migration records; required current
45+
head CI passes. Successful conditional paths require actual enabled fixtures; if
46+
admission is rejected, record the exact sanitized failure and retained test design.
47+
48+
## Evidence
49+
[Verification](../../../evidence/verify/2026-10-19-conditional-acceptance.md).

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/).
3131
- Removed the unconfirmed automatic AK/SK-to-sandbox-key RPC exchange and its private signing/parser helpers. Sandbox requests now fail locally without an API Key; official template/ACR AK/SK operations remain. Explicit custom exchange adapters remain and reject empty/structured token returns. This changes AK/SK-only runtime authentication; see the bilingual migration guide.
3232

3333
### Added
34+
- Add local generation2 pause/snapshot acceptance with memory/files/clone assertions, owned cleanup and bilingual provisioning guidance.
3435

3536
- `Sandbox.create(volume_mounts={"/mnt/data": "existing-volume-name"})` mounts existing Volumes with the built-in provider. Strict validation runs before provider/engine I/O; unsupported providers fail with E8004. The existing gRPC lifecycle forwards the hint and emits the official `volumeMounts: [{name, path}]` creation payload; empty maps preserve prior requests. Real OSS storage mount/read/write/cleanup is verified for the tested base template in cn-hangzhou.
3637
- `agent_sandbox.providers.commands.list_commands`, `get_command` and `run_command` call the shared Node engine for provider-specific resource operations. Writes and deletes require explicit confirmation; command POST requests are never replayed.

‎docs/en/README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -145,3 +145,5 @@ These pages are reviews or target design, not the current specification.
145145
The `abx` CLI (npm `@agent-sandbox/cli`) is documented in the [`cli` repository](https://github.com/alibaba-agent-sandbox/agent-sandbox): [documentation index](https://github.com/alibaba-agent-sandbox/agent-sandbox/blob/main/docs/en/README.md), [Getting Started](https://github.com/alibaba-agent-sandbox/agent-sandbox/blob/main/docs/en/guide/getting-started.md), [CLI Reference](https://github.com/alibaba-agent-sandbox/agent-sandbox/blob/main/docs/en/reference/cli-reference.md), [CLI Architecture](https://github.com/alibaba-agent-sandbox/agent-sandbox/blob/main/docs/en/design/architecture.md), [CLI Command System](https://github.com/alibaba-agent-sandbox/agent-sandbox/blob/main/docs/en/design/cli-design.md), [Templates Catalog](https://github.com/alibaba-agent-sandbox/agent-sandbox/blob/main/docs/en/design/templates-catalog.md), [Deploy & Build](https://github.com/alibaba-agent-sandbox/agent-sandbox/blob/main/docs/en/guide/deploy-and-build.md), [Agent Skill Installation](https://github.com/alibaba-agent-sandbox/agent-sandbox/blob/main/docs/en/guide/agent-skill-installation.md).
146146

147147
The [E2B migration contract](guide/migrate-from-e2b.md) defines synchronous Sandbox, asynchronous AsyncSandbox and the interpreter API; provider/server extensions use ProviderSandbox.
148+
149+
- [Local conditional acceptance](guide/conditional-acceptance.md)
Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# Local Express pause and snapshot acceptance
2+
3+
Keep credentials in ignored local configuration. Native project commands read
4+
ABX_SANDBOX_API_KEY/BASE_URL/DOMAIN; the standard entry additionally maps
5+
E2B_API_KEY/API_URL/DOMAIN. Never send an Express key to another provider.
6+
Official template operations separately need ALICLOUD_ACCESS_KEY_ID and
7+
ALICLOUD_ACCESS_KEY_SECRET; a runtime key cannot replace that authorization.
8+
9+
## Design and prerequisites
10+
11+
Pause/resume and snapshot tests use separate selectors. Prefer an isolated
12+
MicroVM generation2 fixture. Generation1 pause may need enablement; snapshots
13+
require MicroVM and account/region access. Provisioning or admission errors remain
14+
failures, not evidence that a local API key is absent. The public-image fixture
15+
needs no local Docker build or private registry credentials.
16+
17+
## Provision a uniquely named fixture
18+
19+
Choose the same region/Team as the runtime key. Set REGION and TEAM_ID from local
20+
configuration; NAME must be unique and at most36 characters. From a checkout with
21+
the standard CLI entry use this existing project extension:
22+
23+
```bash
24+
REGION=cn-hangzhou
25+
NAME="conditional-$(date +%s)"
26+
abx template from-image \
27+
"fc-e2b-registry.${REGION}.cr.aliyuncs.com/runtime-v2/base:v0.0.53" \
28+
--name "$NAME" --team-id "$TEAM_ID" --region "$REGION" \
29+
--generation 2 --cpu 2 --memory 2048 --disk-size 15360 --no-envd-inject
30+
abx template info "$TEMPLATE_ID"
31+
```
32+
33+
The create result supplies TEMPLATE_ID. Poll info until build state is ready;
34+
creation is asynchronous. Accepted native CLI builds use `template create` for
35+
this image-registration operation; inspect installed help before choosing it.
36+
Do not pass --force or update existing templates for this test.
37+
38+
## Run and verify
39+
40+
Use local environment selectors without committing their values:
41+
42+
```bash
43+
export E2E_PAUSE_TEMPLATE="$TEMPLATE_ID"
44+
export E2E_SNAPSHOT_TEMPLATE="$TEMPLATE_ID"
45+
```
46+
47+
The SDK suite verifies a nonce held only in a running Python process, files,
48+
paused-state observation, connect/resume, snapshot list/restore, clone isolation
49+
and deletion. Paused-list and snapshot indexing are eventually consistent.
50+
Deletion may return409 during pausing/resuming/snapshotting; wait for that known
51+
transition only, with a deadline. Do not retry snapshot creation after an ambiguous
52+
timeout: reconcile snapshots by the uniquely owned source sandbox first.
53+
This backend returned204 even on repeated snapshot deletion, so assert list
54+
absence rather than expecting a second delete to return false.
55+
56+
## Cleanup and evidence
57+
58+
Delete restored/source sandboxes before their occupied snapshot. Delete only
59+
resources created by this run. Verify exact resource absence after cleanup, then
60+
remove the owned fixture with `abx template delete "$TEMPLATE_ID" --yes`.
61+
Record source/artifact SHA, actual passes/failures/skips, admission errors and
62+
owned residual count; omit keys, account identifiers and private endpoint details.
63+
Do not upload credentials or dispatch cloud/model acceptance in Actions.
64+
65+
[Official template setup](https://help.aliyun.com/zh/agent-sandbox/build-and-manage-custom-templates-with-code),
66+
[compatibility](https://www.alibabacloud.com/help/en/agent-sandbox/e2b-compatibility-explanation),
67+
[snapshot limits](https://help.aliyun.com/zh/functioncompute/snapshots).
68+
69+
## SDK test / SDK 测试
70+
71+
```bash
72+
cd e2e
73+
python -m pytest test_standard_contract.py -q -ra
74+
```
75+
76+
E2E_PAUSE_TEMPLATE independently selects pause and can fall back to the snapshot
77+
fixture. E2E_SNAPSHOT_TEMPLATE selects snapshot restoration. Conditional skips
78+
never count as passes. Provisioning and CLI artifact verification belong to
79+
[CLI40](https://github.com/Alibaba-Agent-Sandbox/agent-sandbox/pull/40).
80+
E2E_PAUSE_TEMPLATE 独立选择暂停模板,也可回退到快照模板;
81+
E2E_SNAPSHOT_TEMPLATE 选择快照恢复模板。未配置时基础场景仍执行,条件性跳过不算通过。

‎docs/zh/README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -145,3 +145,5 @@ Agent Sandbox 是可切换提供方的沙箱 SDK。默认提供方是阿里云
145145
`abx` CLI(npm 包 `@agent-sandbox/cli`)的文档位于 [`cli` 仓库](https://github.com/alibaba-agent-sandbox/agent-sandbox):[文档中心](https://github.com/alibaba-agent-sandbox/agent-sandbox/blob/main/docs/zh/README.md)、[快速开始](https://github.com/alibaba-agent-sandbox/agent-sandbox/blob/main/docs/zh/guide/getting-started.md)、[CLI 参考](https://github.com/alibaba-agent-sandbox/agent-sandbox/blob/main/docs/zh/reference/cli-reference.md)、[CLI 架构](https://github.com/alibaba-agent-sandbox/agent-sandbox/blob/main/docs/zh/design/architecture.md)、[CLI 命令体系](https://github.com/alibaba-agent-sandbox/agent-sandbox/blob/main/docs/zh/design/cli-design.md)、[模板目录](https://github.com/alibaba-agent-sandbox/agent-sandbox/blob/main/docs/zh/design/templates-catalog.md)、[部署与构建](https://github.com/alibaba-agent-sandbox/agent-sandbox/blob/main/docs/zh/guide/deploy-and-build.md)、[Agent Skill 安装与分发](https://github.com/alibaba-agent-sandbox/agent-sandbox/blob/main/docs/zh/guide/agent-skill-installation.md)。
146146

147147
[E2B 迁移契约](guide/migrate-from-e2b.md)定义同步 Sandbox、异步 AsyncSandbox 和解释器接口;Provider/Server 扩展使用 ProviderSandbox。
148+
149+
- [本地暂停与快照验收](guide/conditional-acceptance.md)
Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
# 本地极速版暂停与快照验收
2+
3+
凭据保留在被忽略的本地配置中。原生项目入口读取 ABX_SANDBOX_API_KEY、
4+
ABX_SANDBOX_BASE_URL、ABX_SANDBOX_DOMAIN;标准入口还映射 E2B_API_KEY、
5+
E2B_API_URL、E2B_DOMAIN。不要把极速版密钥发给其他提供商。
6+
官方模板操作另需 ALICLOUD_ACCESS_KEY_ID、ALICLOUD_ACCESS_KEY_SECRET,
7+
运行时 API Key 不能代替控制面授权。
8+
9+
## 设计与前置条件
10+
11+
暂停/恢复和快照分别选择测试模板。优先创建隔离的二代 MicroVM 模板;
12+
一代暂停可能需要启用,快照需要 MicroVM 以及账号和地域的相应权限。
13+
构建或准入错误应如实记录,不能归因为缺少本地 API Key。
14+
官方公共镜像无需本地 Docker 构建,也不需要私人仓库凭据。
15+
16+
## 创建独立模板
17+
18+
选择与运行时密钥相同的地域和 Team,TEAM_ID 从本地配置获取。
19+
名称必须唯一且不超过36字符。在具有标准入口的 CLI 检出中执行现有项目扩展:
20+
21+
```bash
22+
REGION=cn-hangzhou
23+
NAME="conditional-$(date +%s)"
24+
abx template from-image \
25+
"fc-e2b-registry.${REGION}.cr.aliyuncs.com/runtime-v2/base:v0.0.53" \
26+
--name "$NAME" --team-id "$TEAM_ID" --region "$REGION" \
27+
--generation 2 --cpu 2 --memory 2048 --disk-size 15360 --no-envd-inject
28+
abx template info "$TEMPLATE_ID"
29+
```
30+
31+
创建结果提供 TEMPLATE_ID。构建是异步操作,轮询 info 直到 ready。
32+
已有原生 CLI 构建以 `template create` 注册镜像模板,须先检查安装版本的帮助。
33+
不传 --force,不修改已有模板。
34+
35+
## 执行与校验
36+
37+
通过本地环境选择模板,不提交真实值:
38+
39+
```bash
40+
export E2E_PAUSE_TEMPLATE="$TEMPLATE_ID"
41+
export E2E_SNAPSHOT_TEMPLATE="$TEMPLATE_ID"
42+
```
43+
44+
SDK 测试校验仅保存在 Python 进程内存中的随机标记、文件、暂停状态、恢复连接、
45+
快照列表、恢复、克隆隔离和删除。暂停状态与快照索引具有最终一致性。
46+
暂停、恢复或快照转换期间删除可能返回409,仅针对已知转换进行有时限的等待。
47+
创建快照超时后不能盲目重试,应按本次独占的源沙箱查询结果。
48+
本次服务对重复删除仍返回204,因此用列表中不存在确认删除,不假设第二次返回 false。
49+
50+
## 清理与证据
51+
52+
先删除恢复和源沙箱,再删除被占用的快照。只清理本轮创建的资源,独立确认不存在后,
53+
用 `abx template delete "$TEMPLATE_ID" --yes` 删除本轮模板。
54+
记录源代码/二进制 SHA、真实通过/失败/跳过和遗留资源数,不记录密钥、账号标识和私人地址。
55+
不向 Actions 上传凭据,不新增或分发云端/模型验收工作流。
56+
57+
[官方模板配置](https://help.aliyun.com/zh/agent-sandbox/build-and-manage-custom-templates-with-code)、
58+
[兼容边界](https://www.alibabacloud.com/help/en/agent-sandbox/e2b-compatibility-explanation)、
59+
[快照限制](https://help.aliyun.com/zh/functioncompute/snapshots)。
60+
61+
## SDK test / SDK 测试
62+
63+
```bash
64+
cd e2e
65+
python -m pytest test_standard_contract.py -q -ra
66+
```
67+
68+
E2E_PAUSE_TEMPLATE independently selects pause and can fall back to the snapshot
69+
fixture. E2E_SNAPSHOT_TEMPLATE selects snapshot restoration. Conditional skips
70+
never count as passes. Provisioning and CLI artifact verification belong to
71+
[CLI40](https://github.com/Alibaba-Agent-Sandbox/agent-sandbox/pull/40).
72+
E2E_PAUSE_TEMPLATE 独立选择暂停模板,也可回退到快照模板;
73+
E2E_SNAPSHOT_TEMPLATE 选择快照恢复模板。未配置时基础场景仍执行,条件性跳过不算通过。

0 commit comments

Comments
 (0)