An AI agent that rewrites TypeScript Selenium test suites in Playwright, and proves the result compiles before you see it.
▶ Try it — varun-s2p.fly.dev
No signup, nothing to install. Paste one Selenium file and get Playwright back, checked. The front page has a four-minute demo if you would rather watch first.
Whole suites run on your own machine. Converting a folder is dozens of model calls in one click, and the public demo pays for its own tokens, so suite conversion is closed there. Clone the repo and run s2p suite with your own API key (Run it yourself). It is the same agent with the same checks.
| You give it Selenium | You get back Playwright |
|---|---|
await driver.findElement(By.id('email')).sendKeys('me@example.com');
await driver.findElement(By.css('button[type=submit]')).click();
await driver.wait(until.urlContains('/home'), 5000); |
await page.locator('#email').fill('me@example.com');
await page.locator('button[type=submit]').click();
await expect(page).toHaveURL(/\/home/); |
- The model can't lie to the compiler. Every conversion must pass four deterministic gates:
tsc --noEmit, typed ESLint, a Selenium-residue scan, and structure parity (same test cases, same assertion coverage as the source). - It fixes its own work. An AI critic reviews the output and the graph repairs what failed, up to three attempts, before anything reaches you. Whatever can't be verified ships as an explicit
TODO(review), never silently. - It converts whole suites. Page objects first, then the specs that import them, in parallel, and then the delivered tree is compiled as one project with a parity ledger you can hand over.
- It asks before it guesses. Three Selenium patterns have more than one correct Playwright translation (dialogs,
executeScript, a session shared by abeforehook). Run it yourself with--threadand the run suspends and asks you, then remembers the answer. The public playground never pauses a stranger mid-run: it applies the playbook's default and names the choice on the result. - It spends tokens only where a model is needed. The agent is never pointed at your repository and never explores it. A scan decides which files actually use Selenium; helpers, fixtures and config are copied across untouched, with no model call at all. Each call carries one Selenium file, the page objects it imports, and — in a suite — a capped slice of the files that call it (most relevant first, 128 KB in total, 32 KB per file), so names match without shipping the tree. Four of the five checks are a compiler, a linter and two AST scripts: they cost nothing to run, however many times the loop repeats.
- It learns your conventions. Tell it once ("wrap each action in a named
test.step()") and it applies that rule in later conversations, on the files where it matters. - It doesn't take orders from the file. A pasted file that isn't Selenium, or that carries text aimed at the model (in a comment, a string, look-alike letters or invisible characters), is refused before any model sees it. The model is told the file is data, and a conversion that loads anything its source never did fails parity.
| What was measured | Result |
|---|---|
| The 12-file sample suite, converted in one run | 12/12 passed all four checks and the critic, and the tree compiles as one project, in about 20 seconds. These are static checks, not a browser run. |
| Does self-correction earn its cost? (12 pinned files, same Opus critic) | Haiku 2/12 → 9/12, the loop's clearest win. Opus 6/12 → 11/12, but only +1 of that is the loop: 4 rows were the critic changing its verdict between runs. Sonnet went 11/12 → 10/12. One run per arm, so a one- or two-file difference is noise. |
| Style, scored by a calibrated LLM judge | Two judges agreed exactly on 81% of rows and within one point on all of them. Both scored Opus's reflected drafts lower than its first drafts. |
| The twelve SDET hard cases, as a second benchmark (GPT-5.4) | 6/11 → 9/11 after two playbook rules. Two identical baselines scored 7/11 and 6/11, so the count alone is weak evidence; the stronger evidence is that the code changed exactly as each rule predicted. |
| Saved conversions replayed in a real browser, with no model spend | 20/20 test rows pass when handed the golden page objects. Page-object rows fail on names a single file can't know, which a suite run fixes. The golden fixtures replay in CI on every push. |
| Offline test suite | 790+ tests on every push, with no secrets and no tokens |
Each result names the model it was measured on. The public playground names its model on every result.
Full numbers and how they were produced: live evaluation page · reflection A/B · judge calibration · hard cases · browser replay · what is still unsolved · how the evals were designed, before they ran.
Fair question. Claude Code can convert a Selenium file, and for one file it does it well. The difference is what you can trust when nobody reviews every file: when the migration is repeated, large, or has to be right.
| Claude Code in the repo | This agent | |
|---|---|---|
| Checking the output | Runs tsc or the linter when it decides to, and decides for itself when it is done. | Compile, lint, residue and parity checks run on every attempt. They are steps in the graph, not a choice the model makes. A failure goes back with its findings, at most three times, before you see any code. |
| Tests that go missing | A test or an assertion can vanish in translation and the file still compiles. Someone has to notice. | Test and assertion counts are compared with the source. A mismatch sends the file back for repair, never a silent drop. So does any import the source never had. |
| Quality | Depends on the prompt and the day. Nobody measures it. | Scored on a fixed evaluation set, and a prompt change ships only after an A/B run on that set. CI replays the golden fixtures in a real browser on every push. |
| Scale | File by file, with someone watching each one. | A whole suite in one command: page objects first, then the tests that use them, compiled together as one project. |
| Who can run it | Someone with prompting skill and access to the repo. | Anyone, with the same result: the playground, the CLI, or a CI pipeline. |
| What reaches the model | Whatever is in the files, including text written to steer the model. | Anything that is not Selenium, or that talks to the model, is refused before a model sees it. Every run is metered against a budget. |
For a one-off file, Claude Code in the repo is genuinely fine. An agent earns its place when the job is repeated, large, or needs guarantees. Closing the gap between "the model can do it" and "a system you can trust unattended" is the engineering this project is about.
Inside the convert graph, every file:
intake → recall → risk_review → convert → validate → critic → assemble
▲ │
└── repair (≤ 3) ──────┘
A LangGraph state machine. Before it starts, a pasted file is screened: anything that is not Selenium, or that talks to the model, is refused without a model call. intake classifies the file or refuses it honestly. recall fetches the few remembered preferences that apply. risk_review pauses the run on a pattern with more than one right answer. convert writes Playwright, validate runs the four gates, critic reviews, and the graph loops back with the actual findings until it passes or the attempt cap is hit. assemble always reports the outcome and keeps the latest draft. Suite mode wraps the same graph in a scan, a parallel fan-out per wave, and a whole-tree compile.
Stack: Python · LangGraph · LangSmith · any LangChain chat model (Claude by default, OpenAI verified end to end, swappable per run with --model) · pinned TypeScript toolchain as the referee · React + Vite playground over FastAPI, self-hosted on Fly behind auth and a dollar budget.
Architecture and decisions · Build log, phase by phase
git clone https://github.com/varunbhatt2193/selenium2playwright && cd selenium2playwright
uv sync
(cd sandbox && npm ci) # the pinned TypeScript toolchain the checks run (Node 22)
cp .env.example .env # add ANTHROPIC_API_KEY (or OPENAI_API_KEY + S2P_MODEL=openai:gpt-5.4)
uv run s2p convert samples/selenium-suite/pages/LoginPage.ts
uv run s2p suite samples/selenium-suite --out out/suite
uv run --group ui python -m unittest discover -s tests # the offline test suite; no key, no tokenss2p convert prints a scorecard and a before/after diff, and writes the converted TypeScript to stdout. s2p suite converts a folder and writes the report beside it. Add --model haiku --critic-model opus for a cheap actor with a strong reviewer, --json for the whole outcome as one document, or uv run langgraph dev to step through a run in LangGraph Studio. CLI walkthrough · playground walkthrough · deployment.
Pull requests are welcome, from a typo to a new gate. Start with CONTRIBUTING.md: it covers setup, the test command, the rules you cannot guess from the code, and where the open work is. Issues labelled good first issue are small and self-contained; a wrong conversion is the most useful report you can file.
MIT.
Built in public by Varun Bhatt. The demo converts single files only, metered per visitor and against a shared daily dollar budget, because every conversion is real tokens on a real card. The page shows how many are left today.