Skip to content

Conversation

@mcndt
Copy link

@mcndt mcndt commented Oct 13, 2025

why

  1. In my organisation, we use Vertex AI for authenticated LLM calls using Google Cloud service accounts instead of the Gemini API keys.
  2. I also need control over the Vertex AI location being used for data residency guarantees.

what changed

I updated the typing of ClientOptions to support the full Google GenAI client constructor options, just like the Anthropic and OpenAI client options are already passed verbatim.

Because the Google GenAI client options don't have a baseUrl key, I also added explicit typing for the client options used for the ai-sdk model provider.

test plan

As this is only a typing change, I foresaw no immediate need for unit tests. Authentication with VertexAI is already tested as part of the GenAI client package.

tkattkat and others added 30 commits September 10, 2025 13:40
# why

solves browserbase#1060 
patch regression of playwright arguments being removed from agent
execute response

# what changed

agent.execute now returns playwright arguments in its response 

# test plan

tested locally
…ms to docs (browserbase#1065)

# why

reflect project id changes in docs

# what changed

advanced configuration comments

# test plan

reviewed via mintlify on localhost
# why

Easier to use for Custom LLM Clients and keep users up to date with our
aisdk file

# what changed

added export of aisdk to lib/index.ts

# test plan

build local stagehand, import local AISdkClient, run Azure Stagehand
session
…onfigu… (browserbase#1073)

…ration settings

# why

Updated docs to match the new fingerprint params in the Browserbase docs
here:
https://docs.browserbase.com/guides/stealth-customization#customization-options

# what changed

Update browser configuration docs to reflect the docs changes. 

# test plan
# why

Updating docs to reflect aisdk can be imported directly

# what changed

The model page

# test plan

Reviewed page with mintlify dev locally
# why

# what changed

# test plan
# why

Currently, we do not support stagehand agent within the api

# what changed

When api is enabled, stagehand agent now routes through the api 

# test plan

Tested locally
# why

Currently, using playwright screenshot command is not available when the
execution environment is Stagehand. A customer has indicated they would
prefer to use Playwright's native screenshot command instead of CDP when
using Browserbase as CDP screenshot causes unexpected behavior for their
target site.

# what changed

- added a StagehandScreenshotOptions type with useCDP argument added
- extended page type to accept custom stagehand screeenshot options
- update screenshot proxy to default useCDP to true if the env is
browserbase and use playwright screenshot if false
- added eval for screenshot with and without cdp

# test plan
- tested and confirmed functionality with eval and external example
script (not committed)
…rowserbase#1057)

# why

We want to build a best in class agent in stagehand.
Therefore, we need more eval benchmarks.

# what changed
- Added Web-bench evals dataset
- Added a subset of OS World evals - those that can be run in a chrome
browser (desktop-based tasks omitted)
- added LICENSE noticed to the copied evals tasks
- Added ground truth / expected result to some WebVoyager tasks using
reference_answer.json from Browser Use public evals repo.

Improvements to `pnpm run evals -man` to better describe how to run
evals.

# test plan
Evals should run locally and bb for these new benchmarks.
# why
Initial instructions didn't mention uv or pip prerequisites and also
didn't mention venv. Fix reduces friction on first timers.

# what changed
- added link to install uv
- added details for initializing venv
- adjusted code example respectively 

# test plan
docs change
# why
- webpage structure changed, needed to update the xpath in the expected
locator
… with LanguageModelV1 + LiteLLM works for python (browserbase#1086)

# why

1. aisdk not yet available through npm package
2. customLLM provider only works with LanguageModelV1
3. LiteLLM compatible providers are supported in python

# what changed

1. change docs to install stagehand from git repo
2. pin versions that use LanguageModelV1

# test plan

local test
# why

currently we pass stagehand page to agent, this results in our page
management having issues when facing new tabs

# what changed

the stagehand object is now passed instead of stagehandPage

# test plan

tested locally
# why

Our existing screenshot service is a dummy time-based triggered service.
It also does not trigger based on any actions of the agent.

# what changed
Added img hash diff algo (quick check with MSE, verify with SSIM algo)
to see if there was an actual UI change and only store ss in the buffer
if that is so.

Added ss interceptor which copies each screenshot the agent is taking to
a buffer (if different enough from the previous ss) to be later used for
evals.

- There's also a small refactor of the agent initialization config to
enable the screenshot collector service to be attached

# test plan
Tests pass locally

---------

Co-authored-by: Miguel <[email protected]>
Co-authored-by: miguel <[email protected]>
# why
To help make sense of eval test cases and results

# what changed
Added metadata to eval runs, cleaned deprecated code

# test plan
# why

# what changed

# test plan
# why

anthropic released a new sota computer use model

# what changed

added claude-sonnet-4-5-20250929 as a model to the list

# test plan

ran evals
…ase#1103)

Why

Custom AI SDK tools and MCP integrations weren't working properly with
Anthropic CUA - parameters were empty {} and tools weren't tracked.

What Changed


- Convert Zod schemas to JSON Schema before sending to Anthropic (using
zodToJsonSchema)
- Track custom tool calls in the actions array
- Silence "Unknown tool name" warnings for custom tools

Test Plan

Tested with examples file. 

Parameters passed correctly ({"city":"San Francisco"} instead of {})
Custom tools execute and appear in actions array
No warnings
# why
To improve context

# what changed
Added current page and url to the system prompt

# test plan
# why
To inform the user throughout the agent execution process

# what changed
Added logs to tool calls, and on the stagehand agent handler

# test plan
- [x] tested locally
PR to make clearer the dependencies for `extract` (for those who haven't
used zod or pydantic before)

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
# why
- before this change, when we convert `z.string().url()` to an ID, if it
was inside a `z.array()`, it was not getting converted back into a URL
- this meant that if you defined a schema like this:
```ts
schema: z.object({
  records: z.array(z.string().url()),
})
```
you would receive an array like this:
```
{
  records: [
    '0-302', '0-309',
    '0-316', '0-323',
    '0-330', '0-337',
    '0-344', '0-351',
    '0-358', '0-365'
  ]
}
```
- with this change, you will now receive the actual URLs, ie:
```
{
  records: [
    'https://www.archives.gov/files/research/jfk/releases/2025/0318/104-10003-10041.pdf',
    'https://www.archives.gov/files/research/jfk/releases/2025/0318/104-10004-10143%20(C06932208).pdf',
    'https://www.archives.gov/files/research/jfk/releases/2025/0318/104-10004-10143.pdf',
    'https://www.archives.gov/files/research/jfk/releases/2025/0318/104-10004-10156.pdf',
    'https://www.archives.gov/files/research/jfk/releases/2025/0318/104-10004-10213.pdf',
    'https://www.archives.gov/files/research/jfk/releases/2025/0318/104-10005-10321.pdf',
    'https://www.archives.gov/files/research/jfk/releases/2025/0318/104-10006-10247.pdf',
    'https://www.archives.gov/files/research/jfk/releases/2025/0318/104-10007-10345.pdf',
    'https://www.archives.gov/files/research/jfk/releases/2025/0318/104-10009-10021.pdf',
    'https://www.archives.gov/files/research/jfk/releases/2025/0318/104-10009-10222.pdf'
  ]
}
```

# what changed
- updated the `injectUrls` function so that when it hits an array and
there is not deeper path, it loops through the array and injects the
URLs
# test plan
- evals
# why
Adding support for Gemini's new Computer Use model

# what changed
We partnered with Google Deepmind to help integrate and test their new
Computer Use models.

<img width="1238" height="655" alt="Screenshot 2025-10-07 at 1 14 44 PM"
src="https://github.com/user-attachments/assets/af0d854a-8e55-4937-a071-10335497f686"
/>

The new model tag `gemini-2.5-pro-computer-use-preview-10-2025` is
available for Stagehand Agent. You can try it today with the example
`cua-example.ts`

To learn more, check out the blog post
[https://www.browserbase.com/blog/evaluating-browser-agents](https://www.browserbase.com/blog/evaluating-browser-agents)

---------

Co-authored-by: tkattkat <[email protected]>
Co-authored-by: Kylejeong2 <[email protected]>
Co-authored-by: Sameel <[email protected]>
# why

# what changed

# test plan
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and the packages will
be published to npm automatically. If you're not ready to do a release
yet, that's fine, whenever you add more changesets to main, this PR will
be updated.


# Releases
## @browserbasehq/[email protected]

### Patch Changes

- [browserbase#1082](browserbase#1082)
[`8c0fd01`](browserbase@8c0fd01)
Thanks [@tkattkat](https://github.com/tkattkat)! - Pass stagehand object
to agent instead of stagehand page

- [browserbase#1104](browserbase#1104)
[`a1ad06c`](browserbase@a1ad06c)
Thanks [@miguelg719](https://github.com/miguelg719)! - Fix logging for
stagehand agent

- [browserbase#1066](browserbase#1066)
[`9daa584`](browserbase@9daa584)
Thanks [@tkattkat](https://github.com/tkattkat)! - Add playwright
arguments to agent execute response

- [browserbase#1077](browserbase#1077)
[`7f38b3a`](browserbase@7f38b3a)
Thanks [@tkattkat](https://github.com/tkattkat)! - adds support for
stagehand agent in the api

- [browserbase#1032](browserbase#1032)
[`bf2d0e7`](browserbase@bf2d0e7)
Thanks [@miguelg719](https://github.com/miguelg719)! - Fix for zod peer
dependency support

- [browserbase#1014](browserbase#1014)
[`6966201`](browserbase@6966201)
Thanks [@tkattkat](https://github.com/tkattkat)! - Replace operator
handler with base of new agent

- [browserbase#1089](browserbase#1089)
[`536f366`](browserbase@536f366)
Thanks [@miguelg719](https://github.com/miguelg719)! - Fixed info logs
on api session create

- [browserbase#1103](browserbase#1103)
[`889cb6c`](browserbase@889cb6c)
Thanks [@tkattkat](https://github.com/tkattkat)! - patch custom tool
support in anthropic cua client

- [browserbase#1056](browserbase#1056)
[`6a002b2`](browserbase@6a002b2)
Thanks [@chrisreadsf](https://github.com/chrisreadsf)! - remove need for
duplicate project id if already passed to Stagehand

- [browserbase#1090](browserbase#1090)
[`8ff5c5a`](browserbase@8ff5c5a)
Thanks [@miguelg719](https://github.com/miguelg719)! - Improve failed
act error logs

- [browserbase#1014](browserbase#1014)
[`6966201`](browserbase@6966201)
Thanks [@tkattkat](https://github.com/tkattkat)! - replace operator
agent with scaffold for new stagehand agent

- [browserbase#1107](browserbase#1107)
[`3ccf335`](browserbase@3ccf335)
Thanks [@seanmcguire12](https://github.com/seanmcguire12)! - fix: url
extraction not working inside an array

- [browserbase#1102](browserbase#1102)
[`a99aa48`](browserbase@a99aa48)
Thanks [@miguelg719](https://github.com/miguelg719)! - Add current page
and date context to agent

- [browserbase#1110](browserbase#1110)
[`dda52f1`](browserbase@dda52f1)
Thanks [@miguelg719](https://github.com/miguelg719)! - Add support for
new Gemini Computer Use models

## @browserbasehq/[email protected]

### Minor Changes

- [browserbase#1057](browserbase#1057)
[`b7be89e`](browserbase@b7be89e)
Thanks [@filip-michalsky](https://github.com/filip-michalsky)! - added
web voyager ground truth (optional), added web bench, and subset of
OSWorld evals which run on a browser

### Patch Changes

- [browserbase#1072](browserbase#1072)
[`dc2d420`](browserbase@dc2d420)
Thanks [@filip-michalsky](https://github.com/filip-michalsky)! - improve
evals screenshot service - add img hashing diff to add screenshots and
change to screenshot intercepts from the agent

- Updated dependencies
\[[`8c0fd01`](browserbase@8c0fd01),
[`a1ad06c`](browserbase@a1ad06c),
[`9daa584`](browserbase@9daa584),
[`7f38b3a`](browserbase@7f38b3a),
[`bf2d0e7`](browserbase@bf2d0e7),
[`6966201`](browserbase@6966201),
[`536f366`](browserbase@536f366),
[`889cb6c`](browserbase@889cb6c),
[`6a002b2`](browserbase@6a002b2),
[`8ff5c5a`](browserbase@8ff5c5a),
[`6966201`](browserbase@6966201),
[`3ccf335`](browserbase@3ccf335),
[`a99aa48`](browserbase@a99aa48),
[`dda52f1`](browserbase@dda52f1)]:
    -   @browserbasehq/[email protected]

## @browserbasehq/[email protected]

### Patch Changes

- Updated dependencies
\[[`8c0fd01`](browserbase@8c0fd01),
[`a1ad06c`](browserbase@a1ad06c),
[`9daa584`](browserbase@9daa584),
[`7f38b3a`](browserbase@7f38b3a),
[`bf2d0e7`](browserbase@bf2d0e7),
[`6966201`](browserbase@6966201),
[`536f366`](browserbase@536f366),
[`889cb6c`](browserbase@889cb6c),
[`6a002b2`](browserbase@6a002b2),
[`8ff5c5a`](browserbase@8ff5c5a),
[`6966201`](browserbase@6966201),
[`3ccf335`](browserbase@3ccf335),
[`a99aa48`](browserbase@a99aa48),
[`dda52f1`](browserbase@dda52f1)]:
    -   @browserbasehq/[email protected]

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
# why

The original example used JavaScript destructuring syntax [table] which
doesn't work in Python. Fixed to use proper Python array indexing.

# what changed

fixed example to proper python syntax

# test plan

Co-authored-by: Steven Bryan <[email protected]>
# why
- need to set default viewport when running on browserbase. previously,
we only defined the default inside the exported `StagehandConfig`
# what changed
- set default viewport to 1288 * 711 when running on browserbase
# test plan
- tested locally,
- regression evals
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and the packages will
be published to npm automatically. If you're not ready to do a release
yet, that's fine, whenever you add more changesets to main, this PR will
be updated.


# Releases
## @browserbasehq/[email protected]

### Patch Changes

- [browserbase#1114](browserbase#1114)
[`c0fbc51`](browserbase@c0fbc51)
Thanks [@seanmcguire12](https://github.com/seanmcguire12)! - configure
default viewport when running on browserbase

## @browserbasehq/[email protected]

### Patch Changes

- Updated dependencies
\[[`c0fbc51`](browserbase@c0fbc51)]:
    -   @browserbasehq/[email protected]

## @browserbasehq/[email protected]

### Patch Changes

- Updated dependencies
\[[`c0fbc51`](browserbase@c0fbc51)]:
    -   @browserbasehq/[email protected]

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@changeset-bot
Copy link

changeset-bot bot commented Oct 13, 2025

🦋 Changeset detected

Latest commit: 108b73a

The changes in this PR will be included in the next version bump.

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Greptile Overview

Summary

This PR updates the Google GenAI client to accept the full GoogleGenAIOptions type instead of a simplified ClientOptions interface, enabling Vertex AI authentication and region control.

Key Changes:

  • Updated ClientOptions type in types/model.ts to include GoogleGenAIClientOptions as a union member
  • Added intersection type with aisdk options (apiKey?, baseURL?) to support AI SDK language model providers
  • Modified GoogleClient.ts to import GoogleGenAIOptions directly from @google/genai and pass the full options object to the client constructor
  • Implementation now matches the pattern used by AnthropicClient and OpenAIClient, which pass their full client options verbatim

Potential Concern:
The intersection type on lines 48-57 of types/model.ts adds apiKey?: string and baseURL?: string to all client options. If GoogleGenAIOptions already defines these properties with different types, this could cause type conflicts. However, since the PR author states this is a typing-only change with no runtime impact, this suggests the properties are compatible.

Confidence Score: 4/5

  • This PR is safe to merge with minimal risk - it's a type-only change that expands functionality without breaking existing code.
  • Score reflects the low-risk nature of the change (type definitions only), proper documentation via changeset, and consistent pattern matching with other client implementations. One point deducted due to potential type intersection conflicts with the added apiKey? and baseURL? properties in the union type - while likely compatible, this wasn't explicitly verified in the PR.
  • Pay attention to types/model.ts - verify that the intersection type with aisdk options doesn't conflict with existing GoogleGenAIOptions properties.

Important Files Changed

File Analysis

Filename Score Overview
types/model.ts 4/5 Updated ClientOptions type to include full GoogleGenAIClientOptions and added intersection with aisdk options (apiKey, baseURL). The intersection type may cause type conflicts if GoogleGenAIOptions already defines these properties.
lib/llm/GoogleClient.ts 5/5 Switched from local ClientOptions import to GoogleGenAIOptions directly from @google/genai. Changed constructor to pass full clientOptions to GoogleGenAI instead of just apiKey. Clean implementation that matches the pattern used by Anthropic and OpenAI clients.

Sequence Diagram

sequenceDiagram
    participant User
    participant Stagehand
    participant LLMProvider
    participant GoogleClient
    participant GoogleGenAI

    User->>Stagehand: Initialize with modelClientOptions
    Stagehand->>LLMProvider: getClient(modelName, clientOptions)
    LLMProvider->>GoogleClient: new GoogleClient({clientOptions})
    Note over GoogleClient: clientOptions now supports full GoogleGenAIOptions
    GoogleClient->>GoogleClient: Check for apiKey in clientOptions
    alt apiKey missing
        GoogleClient->>GoogleClient: loadApiKeyFromEnv()
    end
    GoogleClient->>GoogleGenAI: new GoogleGenAI(clientOptions)
    Note over GoogleGenAI: Full options passed (apiKey, location, project, etc.)
    GoogleGenAI-->>GoogleClient: client instance
    GoogleClient-->>LLMProvider: GoogleClient instance
    LLMProvider-->>Stagehand: LLMClient
    User->>Stagehand: createChatCompletion()
    Stagehand->>GoogleClient: createChatCompletion()
    GoogleClient->>GoogleGenAI: models.generateContent()
    GoogleGenAI-->>GoogleClient: response
    GoogleClient-->>Stagehand: LLMResponse
    Stagehand-->>User: result
Loading

2 files reviewed, no comments

Edit Code Review Agent Settings | Greptile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.