Annotations for Scrollable elements in a11y-dom hybrid

[seanmcguire12]

why

• we want the LLM to be able to find scrollable containers in observe calls
• we don't have this level of granularity with the a11y tree alone

what changed

• changed the getMainScrollableElement function in process.ts to getScrollableElements, which now optionally returns a list of all the scrollable elements on the page unless the topN param is specified.
• For usage in processAllOfDom, we set topN to 1 to maintain the existing behaviour.
• added a new function in process.ts called getScrollableElementXpaths which calls getScrollableElements and then calls generateXpaths on each of the returned elements. This function returns a list of xpaths for each scrollable element on the page.
• exposed the new getScrollableElementXpaths globally so that it can be called from a11y/utils.ts
• created a new function findScrollableElementIds in a11y/utils.ts which calls the getScrollableElementXpaths function from the browser context, evaluates the returned xpaths, and returns their corresponding backendDOMId's
• updated getAccessibilityTree to prepend the string Scrollable to the role of all the scrollable elements in the a11y tree

example

LLM input before this PR:

LLM input after this PR:

[changeset-bot]

🦋 Changeset detected

Latest commit: a335bca

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@browserbasehq/stagehand	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR