🐛 fix(desktop): 使用真实 CDP 输入并修复搜索框定位 - #268
Open
CavinHuang wants to merge 4 commits into
Open
Conversation
将点击、双击、文本输入和按键从页面内合成事件切换为 Chromium CDP Input 序列。普通页面与 iframe locator 统一先解析稳定坐标，再执行真实输入；表单动作读取最终状态，避免静默报告成功。 Constraint: 不依赖尚未合并的内置 Browser Tools PR Constraint: 保留 select 的标准 DOM 状态设置并验证最终值 Rejected: 继续使用 element.click 与人工 InputEvent | 无法覆盖受控输入和真实用户激活语义 Tested: browser-cdp-input 4 个单元测试通过 Tested: @lume/desktop 类型检查通过
补全可访问名称解析和关联 label 命中，使用可产生文本的原生 CDP Enter 事件，并将 DOM 命中坐标规范为整数。 Constraint: 保持真实 CDP 输入，不回退到 JavaScript 合成事件 Tested: browser-cdp-input 与 browser-locator 8 个单元测试 Tested: @lume/desktop typecheck Not-tested: 完整 Browser E2E 被现有 WebMCP 基线问题 #269 阻断；新增搜索断言在阻断前全部通过
CavinHuang
changed the title
Electron 会丢弃连续路由到子 frame 的 root-session 指针序列，因此对 iframe 内按钮和链接先用 CDP DOM.focus 聚焦，再发送原生 Enter 完成浏览器语义激活。 Constraint: 不使用 element.click 或合成 MouseEvent Tested: browser-cdp-input 与 browser-locator 9 个单元测试 Tested: @lume/desktop typecheck Tested: 合入 #269 修复后 Electron Browser Runtime E2E 70 assertions
Owner
Author
同一 BrowserSession 的变更型请求进入单写者队列。动作真正执行前重新校验标签页身份、generation 与 Agent lease；前一动作导航后，队列中基于旧页面生成的动作返回 stale_target，不再落到新页面。 Constraint: 本提交不实现用户接管暂停与动作后效果等待 Rejected: 仅依赖模型串行发工具 | 并行 tool_use 仍可能同时落到页面 Tested: Browser action queue/CDP/locator 11 tests Tested: Electron Browser Runtime E2E 72 assertions（临时组合 PR #270 后验证） Tested: bun run --filter @lume/desktop typecheck
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
结果
完成 #262 的真实输入纵向切片，补齐 #253 的百度类搜索框链路，并交付 #263 的首个单写者队列切片。Browser locator 动作不再依赖
element.click()、人工 KeyboardEvent 或直接改 value。变更
Input.dispatchMouseEvent的 move、press、release 完整序列Input.insertTextlabel.controlstale_target验证
@lume/desktoptypecheck 通过后续风险
Closes #253
Related: #251 #262 #263 #269 #270