✨ feat(sidecar,desktop): 添加任务级内置浏览器工具与剪枝快照 - #266
Open
CavinHuang wants to merge 4 commits into
Open
Conversation
把任务级 BrowserSession、Agent 标签页锁定和 AX Tree 剪枝快照下沉到宿主工具层，模型无需再通过 Node REPL 编写浏览器控制代码。保留现有 BrowserClient DOMSnapshot 路径，便于后续动作工具渐进迁移。 Constraint: 保留旧 BrowserClient DOMSnapshot 兼容路径 Constraint: 子代理不共享主任务的浏览器控制标签页 Tested: 32 个相关单元测试通过 Tested: @lume/sidecar 与 @lume/desktop 类型检查通过
This was referenced Aug 20, 2026
为内置 Browser Tools 增加 run_script。脚本仅能在当前任务锁定的 Agent 标签页中运行，每次创建全新隔离世界，并通过现有浏览器动作确认策略执行。参数、时间和 JSON 结果均设置宿主边界，页面异常以结构化工具错误返回。 Constraint: 脚本不能控制用户标签页或其他任务标签页 Constraint: 任意脚本执行必须经过浏览器动作确认 Constraint: 返回值必须可 JSON 序列化且不超过 200000 字符 Rejected: 复用 Node REPL 注入 BrowserClient | 继续依赖模型拼装控制代码 Tested: 41 个相关单元测试通过 Tested: @lume/desktop 与 @lume/sidecar 类型检查通过
CavinHuang
changed the title
浏览器工具结果包含 operation id 与 tab id 等易变字段，不能直接用于判断页面是否产生新进展。为 open、switch_tab、run_script 和错误结果提供稳定状态，供 Agent Loop 的重复动作熔断使用。 Constraint: 不改变模型可见的浏览器工具结果 Tested: bun test apps/sidecar/src/services/agent-runtime/tools/browser/create-browser-tools.test.ts Tested: bun run --filter @lume/sidecar typecheck
Owner
Author
普通 Agent 浏览器任务直接使用 mcp__browser__* 工具，并在整条用户请求的内部迭代间复用，不再暴露或要求激活 browser:browser。用户显式引用已有浏览器页面时仍保留旧 claim 流程。 Constraint: 浏览器附件仍依赖旧 Skill 的用户 Tab 授权流程 Rejected: 仅修改 Skill 文案 | 仍会暴露 Node REPL API 猜测路径 Tested: ContextAssembler 20 tests Tested: runtime-core Browser route test Tested: bun run --filter @lume/sidecar typecheck
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
结果
完成任务级内置浏览器控制的首个纵向闭环：模型直接调用 5 个宿主 Browser Tools，不再依赖 Browser Skill 与 Node REPL 编写 BrowserClient 控制代码。
变更
mcp__browser__list_tabs、open、switch_tab、snapshot、run_scriptswitch_tab会显式切换Accessibility.getFullAXTree生成剪枝语义树，可交互节点获得会话内稳定的eN引用run_script只在锁定标签页的全新 Chromium 隔离世界执行，带大小、超时、序列化和确认边界browser:browser；整条用户请求的内部迭代直接复用内置工具验证
@lume/sidecartypecheck 通过@lume/desktoptypecheck 通过后续
这是 #260 与 #261 的首个纵向切片，不自动关闭它们。后续 PR 继续补齐 ref 动作工具、actionability 等待、动作后观察和用户接管状态机。
Closes #255
Related: #251 #254 #260 #261 #262 #263 #272