Skip to content

R&D: browserbase for the web resources...#170

Open
bmdavis419 wants to merge 2 commits intodavis/web-resourcefrom
davis/browserbase-web-resource
Open

R&D: browserbase for the web resources...#170
bmdavis419 wants to merge 2 commits intodavis/web-resourcefrom
davis/browserbase-web-resource

Conversation

@bmdavis419
Copy link
Collaborator

@bmdavis419 bmdavis419 commented Feb 6, 2026

Greptile Overview

Greptile Summary

  • Adds a new workspace package @btca/bb to integrate Browserbase + puppeteer-core as an optional HTML renderer.
  • Updates the server website resource to convert HTML to Markdown via Turndown (GFM plugin) and optionally fall back to the renderer for SPA-like pages.
  • Refactors several Result-handling callsites (config/metrics/git) to use Result.isError/manual throws.
  • Extends website resource args/tests to support an optional renderer and render budget behavior.

Confidence Score: 3/5

  • This PR is moderately safe to merge, but there are a couple of concrete issues in the website renderer integration that should be fixed first.
  • Core changes are additive and tests were expanded, but apps/server/src/resources/impls/website.ts has a broken triple-slash reference path (can break typechecking) and render budget/test-run detection logic that will behave incorrectly in real runs.
  • apps/server/src/resources/impls/website.ts

Important Files Changed

Filename Overview
apps/server/package.json Adds @btca/bb workspace dep plus turndown + gfm plugin for HTML→Markdown conversion in website resource.
apps/server/src/config/config.test.ts Updates config test expectation to use provider 'opencode' for updateModel test case.
apps/server/src/config/index.ts Refactors Result.match error handling to Result.isError branches; behavior should remain the same.
apps/server/src/metrics/index.ts Refactors span() to use Result.isError; should preserve logging/throw semantics.
apps/server/src/resources/impls/git.ts Refactors error propagation from Result.match to local error variable + throw; behavior appears equivalent.
apps/server/src/resources/impls/website.test.ts Adds tests for Turndown markdown conversion and renderer usage/budget; updates base args to include renderer:null.
apps/server/src/resources/impls/website.ts Adds Turndown-based HTML→Markdown and optional Browserbase renderer fallback; includes a broken triple-slash type reference path and render budget decrement before render success.
apps/server/src/resources/types.ts Extends BtcaWebsiteResourceArgs with optional BbRenderer; imports type from new @btca/bb package.
apps/server/src/types/turndown-plugin-gfm.d.ts Adds a local d.ts shim for turndown-plugin-gfm; website.ts currently references it via an incorrect relative triple-slash path.
bun.lock Updates lockfile to include @btca/bb workspace package and new dependencies (browserbase SDK, puppeteer-core, turndown).
package.json Adds dev:bb turbo script for the new @btca/bb package.
packages/bb/.gitignore Adds standard ignores for the new bb package.
packages/bb/README.md Adds minimal README for bb R&D package.
packages/bb/package.json Introduces new private workspace package @btca/bb with browserbase SDK + puppeteer-core deps and a smoke dev script.
packages/bb/src/index.test.ts Adds unit tests for getBrowserbaseEnv() handling missing/present env vars.
packages/bb/src/index.ts Adds Browserbase-backed renderer with serialized renders and lazy init; no obvious runtime breakages but relies on puppeteer request interception details.
packages/bb/src/smoke.ts Adds a smoke script that renders a docs URL via Browserbase and prints HTML snippet.
packages/bb/tsconfig.json Adds strict tsconfig for new bb package.

Copy link
Collaborator Author

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@bmdavis419 bmdavis419 changed the title added in browserbase testing shit R&D: browserbase for the web resources... Feb 6, 2026
@bmdavis419 bmdavis419 marked this pull request as ready for review February 6, 2026 02:39
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

18 files reviewed, 3 comments

Edit Code Review Agent Settings | Greptile

Comment on lines +1 to 3
/// <reference path="../../types/turndown-plugin-gfm.d.ts" />

import { promises as fs } from 'node:fs';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bad type reference path

This /// <reference path="../../types/turndown-plugin-gfm.d.ts" /> points to apps/server/src/types/..., but from apps/server/src/resources/impls/website.ts the relative path should be ../../types/...? Actually resources/impls -> resources -> src so ../../types resolves to apps/server/src/resources/types, not apps/server/src/types. This will break TS type resolution for turndown-plugin-gfm in this file.

Suggested change
/// <reference path="../../types/turndown-plugin-gfm.d.ts" />
import { promises as fs } from 'node:fs';
/// <reference path="../../types/turndown-plugin-gfm.d.ts" />

Adjust the path so it resolves to apps/server/src/types/turndown-plugin-gfm.d.ts (or add this declaration to a location included by TS without a triple-slash reference).

Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/server/src/resources/impls/website.ts
Line: 1:3

Comment:
**Bad type reference path**

This `/// <reference path="../../types/turndown-plugin-gfm.d.ts" />` points to `apps/server/src/types/...`, but from `apps/server/src/resources/impls/website.ts` the relative path should be `../../types/...`? Actually `resources/impls` -> `resources` -> `src` so `../../types` resolves to `apps/server/src/resources/types`, not `apps/server/src/types`. This will break TS type resolution for `turndown-plugin-gfm` in this file.

```suggestion
/// <reference path="../../types/turndown-plugin-gfm.d.ts" />
```
Adjust the path so it resolves to `apps/server/src/types/turndown-plugin-gfm.d.ts` (or add this declaration to a location included by TS without a triple-slash reference).

How can I resolve this? If you propose a fix, please make it concise.

Comment on lines +718 to 720
shell: false
};
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Render budget leaks

args.renderState.remaining -= 1 happens before confirming the render actually succeeded and produced a usable result; if renderer.render() throws or returns a cross-origin finalUrl (you return null), you still burn budget. In crawls with intermittent render failures, this will stop rendering much earlier than intended.

Consider only decrementing remaining after a successful render that passes the origin check (and/or only after you decide to accept the rendered snapshot).

Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/server/src/resources/impls/website.ts
Line: 718:720

Comment:
**Render budget leaks**

`args.renderState.remaining -= 1` happens before confirming the render actually succeeded and produced a usable result; if `renderer.render()` throws or returns a cross-origin `finalUrl` (you return `null`), you still burn budget. In crawls with intermittent render failures, this will stop rendering much earlier than intended.

Consider only decrementing `remaining` after a successful render that passes the origin check (and/or only after you decide to accept the rendered snapshot).

How can I resolve this? If you propose a fix, please make it concise.

Comment on lines 1086 to 1088
});
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test detection can misfire

process.argv.includes('test') will treat any run whose argv contains the string test as a test run and disable renderer auto-loading. That can inadvertently disable rendering in normal execution (e.g. a path containing test or a user-provided arg). Prefer relying on Bun.env.BUN_TEST (or a dedicated config flag) instead of argv substring matching.

Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/server/src/resources/impls/website.ts
Line: 1086:1088

Comment:
**Test detection can misfire**

`process.argv.includes('test')` will treat any run whose argv contains the string `test` as a test run and disable renderer auto-loading. That can inadvertently disable rendering in normal execution (e.g. a path containing `test` or a user-provided arg). Prefer relying on `Bun.env.BUN_TEST` (or a dedicated config flag) instead of argv substring matching.

How can I resolve this? If you propose a fix, please make it concise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Comments