Skip to content

Feature Request: Add frameLocator support for cross-origin iframe interaction #279

@jeffscottward

Description

@jeffscottward

Problem

When trying to interact with elements inside cross-origin iframes (such as Apple's authentication iframe on icloud.com), agent-browser cannot access or interact with the iframe content.

Current Behavior

  • agent-browser snapshot does not include elements from iframes
  • Mouse clicks via agent-browser mouse move/down/up click on the iframe but don't focus internal elements
  • Keyboard input via agent-browser press doesn't reach elements inside the iframe
  • agent-browser fill "iframe >> input" "text" doesn't work for cross-origin iframes
  • agent-browser eval cannot access iframe content due to cross-origin security restrictions

Expected Behavior

Playwright supports iframe interaction via page.frameLocator() which properly handles cross-origin frames:

const frame = page.frameLocator('iframe[name="auth-widget"]');
await frame.locator('input[type="text"]').fill('[email protected]');

Proposed Solution

Add frameLocator support to agent-browser, potentially via:

  1. A new --frame flag for commands:

    agent-browser fill --frame "iframe[name='auth']" "input" "text"
    agent-browser click --frame "iframe" "@e1"
  2. Or a frame selector syntax:

    agent-browser fill "frame:iframe[name='auth'] >> input" "text"
  3. Or frame-specific commands:

    agent-browser frame "iframe[name='auth']" fill "input" "text"

Use Case

This is critical for automating authentication flows (Apple, Google, OAuth providers) where login forms are embedded in cross-origin iframes for security.

Environment

  • agent-browser version: latest (via npx)
  • macOS
  • Tested with iCloud authentication (idmsa.apple.com iframe)

Workaround

Currently, users must use Playwright directly or tools like mcp__next-devtools__browser_eval which properly expose frameLocator functionality.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions