🐛 fix(desktop): 修复 WebMCP 桥接工具调用 - #270
Open
CavinHuang wants to merge 1 commit into
Open
Conversation
使用 contextBridge 暴露的真实注册入口，并在调用时只回传可克隆的工具名称引用，避免将桥接代理对象再次跨上下文传递。 Constraint: 不改变 WebMCP 页面协议和公开工具结构 Tested: Electron Browser Runtime E2E 65 assertions Tested: @lume/desktop typecheck Not-tested: browser-webmcp-consumer 单文件测试受现有 Electron ESM mock 配置阻断
This was referenced Aug 20, 2026
CavinHuang
pushed a commit
that referenced
this pull request
Aug 20, 2026
同一 BrowserSession 的变更型请求进入单写者队列。动作真正执行前重新校验标签页身份、generation 与 Agent lease；前一动作导航后，队列中基于旧页面生成的动作返回 stale_target，不再落到新页面。 Constraint: 本提交不实现用户接管暂停与动作后效果等待 Rejected: 仅依赖模型串行发工具 | 并行 tool_use 仍可能同时落到页面 Tested: Browser action queue/CDP/locator 11 tests Tested: Electron Browser Runtime E2E 72 assertions（临时组合 PR #270 后验证） Tested: bun run --filter @lume/desktop typecheck
CavinHuang
pushed a commit
that referenced
this pull request
Aug 21, 2026
内置浏览器动作不再仅靠 role/name/nth 二次查找，而是携带 snapshotId 与 ref，由桌面运行时解析同一会话 RefMap 中的 backendDOMNodeId。动作前同时校验 Tab、导航代次、可见性、可用性和命中遮挡。 Constraint: 旧 snapshotId 的 ref 必须立即返回 stale_target，不能进入 locator 自动等待 Tested: desktop 与 sidecar TypeScript 类型检查 Tested: semantic snapshot 与 browser tools 单元测试（12 项） Not-tested: 完整 browser-runtime E2E 在新语义 ref 断言全部通过后，停在 PR #270 修复的既有 WebMCP 归一化问题
CavinHuang
pushed a commit
that referenced
this pull request
Aug 21, 2026
遍历 Page frame tree 并按 frameId 获取各 Frame 的 AX Tree，节点 ID 在合并前按 Frame 命名空间隔离，RefMap 同时绑定 frameId 与 backendDOMNodeId。跨域 Frame 的 @en 可直接走精确节点动作。 Constraint: 单个不可访问 Frame 失败时保留其他 Frame 快照 Tested: semantic snapshot、broker 与 browser tools 单元测试（31 项） Tested: desktop、sidecar、shared TypeScript 类型检查 Not-tested: 完整 E2E 已通过跨域 Frame 快照与精确点击断言，随后停在 PR #270 已修复的既有 WebMCP 归一化问题
CavinHuang
pushed a commit
that referenced
this pull request
Aug 21, 2026
在每个 Frame 的隔离世界批量发现 cursor:pointer、onclick、tabindex 与 contenteditable 元素，通过 CDP objectId 解析 backendDOMNodeId，并合并回已有 AX 节点或生成 clickable/focusable 节点。 Constraint: 跳过原生交互标签、交互 ARIA role、隐藏节点和继承 pointer 的子节点 Constraint: 每个 Frame 最多补充 500 个节点且远程对象始终释放 Tested: semantic snapshot 单元测试（4 项）与 desktop TypeScript 类型检查 Not-tested: 完整 E2E 已通过 cursor-pointer 快照与精确点击断言，随后停在 PR #270 已修复的既有 WebMCP 归一化问题
CavinHuang
pushed a commit
that referenced
this pull request
Aug 21, 2026
标注截图复用最新语义快照的稳定引用，并在隔离画布中绘制，不修改页面 DOM。仅标注主 Frame 可见元素，跨 Frame 引用继续保留语义动作能力。`n`nConstraint: 不引入图像处理依赖`nConstraint: 标注截图不能与 full_page 同时使用`nTested: Browser Tools 与 Broker 单元测试（30 项）`nTested: desktop/sidecar typecheck`nTested: Electron 回归两次通过标注截图与同源引用断言`nNot-tested: 完整 Electron 套件仍被 PR #270 修复的既有 WebMCP 归一化失败阻断
CavinHuang
pushed a commit
that referenced
this pull request
Aug 21, 2026
scope_ref 复用宿主缓存的同一份语义快照，不重新抓取页面，不失效当前 refs，也不增加 CDP 观察压力。cursor 继续对切片后的宿主树分页。`n`nConstraint: scope_ref 必须绑定当前 snapshot_id`nRejected: 每次 scope 重新抓取 AX Tree | 连续观察会增加 CDP 抖动并意外失效 refs`nTested: 快照、Browser Tools 与 Broker 相关测试（35 项）`nTested: desktop/sidecar typecheck`nTested: Electron 回归通过 scope 子树与标注截图断言，继续停在 PR #270 对应的既有 WebMCP 用例
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
结果\n\n修复 #269：页面通过 preload/contextBridge 暴露的真实 WebMCP shim 注册后，可以稳定 list 和 invoke。\n\n## 根因\n\n- E2E fixture 给 document.modelContext 赋值，没有覆盖 consumer 优先读取的 window.__lumeWebMcpModelContext。\n- invoke 将 getTools() 返回的 contextBridge 代理对象原样传回 executeTool，Electron 无法稳定克隆该对象，最终被折叠为 browser_internal_error。\n\n## 变更\n\n- fixture 使用 window.__lumeWebMcpModelContext.registerTool 注册真实工具\n- invoke 只传递可克隆的普通引用 { name }\n- consumer 测试加入额外公开字段，锁定这些字段不会被原样回传\n- 删除 fixture 自制的 modelContext/getTools/executeTool 替身\n\n## 验证\n\n- Electron Browser Runtime E2E：65 assertions 全部通过\n- @lume/desktop typecheck 通过\n- browser-webmcp-consumer 单文件测试仍受仓库现有 Electron ESM mock 配置阻断，与本次改动无关\n\nCloses #269\nRelated: #251 #260