feat: implement frameLocator support for cross-origin iframe interaction#1030
feat: implement frameLocator support for cross-origin iframe interaction#1030giulio-leone wants to merge 4 commits intovercel-labs:mainfrom
Conversation
|
@giulio-leone is attempting to deploy a commit to the Vercel Labs Team on Vercel. A member of the Team first needs to authorize it. |
…ion (vercel-labs#279) Implement the frameLocator feature request from issue vercel-labs#279, enabling cross-origin iframe interaction via CDP execution context scoping. Core changes: - Add context_id to EvaluateParams for frame-scoped JS evaluation - Add BrowserManager::get_frame_context_id() using Page.createIsolatedWorld - Add BrowserManager::evaluate_in_context() for frame-scoped evaluation - Add frame locator CLI command: 'frame locator <selector>' / 'frame locator clear' - Add handle_framelocator() action handler with set/clear semantics Frame-aware execution: - handle_evaluate() now scopes evaluation to active frame - handle_getbyrole() locates elements within frame context - handle_semantic_locator() (getByText, getByLabel, etc.) frame-aware - execute_subaction() resolves clicks/fills/hovers within iframe coordinates - Viewport-absolute coordinate calculation for cross-origin iframe clicks Improved frame resolution: - Multi-strategy matching: name -> id -> src URL -> nth child frame - Extracted find_frame_in_tree() and find_nth_child_frame() helpers - Handles anonymous iframes without name/id attributes
- Fix resolve_element_in_frame to target the specific iframe via stored selector instead of grabbing the first visible iframe on the page - Reorder frame matching to prefer src URL over id-as-name fallback, since CDP's name field maps to iframe.name, not iframe.id - Run cargo fmt for consistent formatting Co-authored-by: Copilot <[email protected]>
- Run cargo fmt across 4 files (commands.rs, element.rs, snapshot.rs, storage.rs) - Refactor active_frame_selector assignment to use Option::map/or_else instead of manual if-let chain (actions.rs:3590)
Co-authored-by: Copilot <[email protected]>
fe4b79d to
dd96ebe
Compare
|
Rebased this branch onto current While validating the rebased diff I found and fixed one real regression in the new Local Rust validation passed on the pushed head:
GitHub has now rerun |
Summary
Implements the
frameLocatorfeature request from #279, enabling cross-origin iframe interaction (Apple auth, OAuth providers, embedded forms, etc.) via CDP execution context scoping.Supersedes #460 (which targeted the old TypeScript codebase).
Changes
Core: Frame-scoped evaluation
context_id: Option<i64>toEvaluateParamsfor frame-scopedRuntime.evaluateBrowserManager::get_frame_context_id()usingPage.createIsolatedWorldwithgrantUniversalAccess: trueBrowserManager::evaluate_in_context()for optional frame-scoped JS evaluationNew command:
frame locatorframe locator <css-selector>— scope all subsequent commands to the matched iframeframe locator clear— return to main frame contextpage.frameLocator(selector)Frame-aware command execution
evaluate— JS evaluation scoped to active framegetByRole,getByText,getByLabel,getByPlaceholder,getByTestId,getByTitle,getByAltText— semantic locators resolve elements within iframe DOMclick,fill,check,hover,textsubactions — coordinate resolution accounts for iframe viewport offsetdata-agent-browser-located) runs in the correct frame contextImproved frame resolution
nameoridattributesfind_frame_in_tree()andfind_nth_child_frame()as reusable helpersHow it works
When
frame locator <selector>is invoked:name,id, andsrcare extractedPage.getFrameTree) is searched using multiple strategiesPage.createIsolatedWorldcreates an execution context in the target framegetBy*,evaluate, and interaction commands use this contextframe locator clearresets to the main frameThis uses CDP's built-in cross-origin access (
grantUniversalAccess), bypassing same-origin restrictions at the browser level — no proxy or content injection needed.Testing
Closes #279