Skip to content

fix: resolve leaked extract() element IDs on z.string() URL fields - #2784

Open
antonvishal wants to merge 8 commits into
browserbase:mainfrom
antonvishal:fix/extract-string-url-ids
Open

antonvishal wants to merge 8 commits into
browserbase:mainfrom
antonvishal:fix/extract-string-url-ids

Conversation

@antonvishal

@antonvishal antonvishal commented Aug 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • extract() leaked accessibility-tree IDs ([0-74], 0-74) when the schema used plain z.string() for URL fields. injectUrls only ran for z.url() / format: uri.
  • Agents and callers that extract links as strings (for example { stories: z.string() } or { rank1_url: z.string(), ... } on a page like Hacker News) got raw element IDs back. z.url() fields already resolved correctly.
  • Fix: tell the model to return frameId-backendId only for URL values, then replaceElementIdsWithUrls maps leaked IDs through combinedUrlMap. Bracketed IDs rewrite in any string; bare IDs only on URL-ish field names.

Before

{ "stories": "[0-74]\n[0-112]\n[0-150]\n[0-188]\n[0-226]" }

{
"rank1_title": "GrapheneOS in 2027 available on high-end Motorola phones",
"rank1_url": "0-74",
"rank2_url": "0-112",
"rank3_url": "0-150",
"rank4_url": "0-188",
"rank5_url": "0-226"
}

After (this fix)

{
"stories": "https://grapheneos.social/@GrapheneOS/117078064184215730\nhttps://yassa9.github.io/osint/gralhix-004/\nhttps://www.raphaelbauer.com/posts/postgresql-everything/\nhttps://sprocketfox.io/xssfox/2026/08/19/sondehub-and-war/\nhttps://openlogi.org/en"
}

{
"rank1_title": "GrapheneOS in 2027 available on high-end Motorola phones",
"rank1_url": "https://grapheneos.social/@GrapheneOS/117078064184215730",
"rank2_url": "https://yassa9.github.io/osint/gralhix-004/",
"rank3_url": "https://www.raphaelbauer.com/posts/postgresql-everything/",
"rank4_url": "https://sprocketfox.io/xssfox/2026/08/19/sondehub-and-war/",
"rank5_url": "https://openlogi.org/en"
}

Test plan

  • pnpm exec vitest run --root . packages/extension/tests/extract.test.ts
  • extract() on a page with links using z.object({ title: z.string(), url: z.url() }) still returns real hrefs
  • Same instruction with z.object({ title: z.string(), url: z.string() }) returns real hrefs, not element IDs
  • extract("Extract the first 5 story URLs", z.object({ urls: z.string() })) returns real URLs, not [0-74] / 0-74
  • Non-URL strings that look like IDs (e.g. score: "0-74") are left unchanged

Summary by cubic

Fixes extract() leaking accessibility-tree element IDs when URL fields use z.string(). Only z.url() fields resolved to hrefs before; now plain string URL fields also return real URLs, and non-URL fields never return element IDs.

  • Updates the extraction instruction so URL values return the link ID as frameId-backendId (no brackets) and all other fields return only visible text.
  • Adds replaceElementIdsWithUrls to map leaked IDs to hrefs via the snapshot combinedUrlMap: bracketed IDs are replaced anywhere, bare IDs only on URL-like field names, and unknown IDs are left as-is.
  • Extends transformSchema to treat url, uri, and uri-reference as URL formats, preserving the z.url() path.
  • Regenerates the Go SDK extension asset so the fix ships there too.

Migration

  • Callers relying on raw element IDs in string fields should update that logic to accept real URLs.

Written for commit 6c77b7e. Summary will update on new commits.

View guided diff

@changeset-bot

changeset-bot Bot commented Aug 20, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: e742ee9

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@github-actions

Copy link
Copy Markdown
Contributor

This PR is from an external contributor and must be approved by a stagehand team member with write access before CI can run.
Approving the latest commit mirrors it into an internal PR owned by the approver.
If new commits are pushed later, the internal PR stays open but is marked stale until someone approves the latest external commit and refreshes it.

@github-actions github-actions Bot added external-contributor Tracks PRs mirrored from external contributor forks. external-contributor:awaiting-approval Waiting for a stagehand team member to approve the latest external commit. labels Aug 20, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external-contributor:awaiting-approval Waiting for a stagehand team member to approve the latest external commit. external-contributor Tracks PRs mirrored from external contributor forks.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant