Skip to content

feat(parser): static fiber reconstruction from source ASTs with runtime verification harness - #115

Open
aidenybai wants to merge 10 commits into
mainfrom
devin/1788659752-parser-package
Open

feat(parser): static fiber reconstruction from source ASTs with runtime verification harness#115
aidenybai wants to merge 10 commits into
mainfrom
devin/1788659752-parser-package

Conversation

@aidenybai

@aidenybai aidenybai commented Sep 6, 2026

Copy link
Copy Markdown
Owner

Summary

Adds packages/parser (@bippy/parser, private): static reconstruction of React fiber trees from source ASTs, plus a harness that captures the real tree through bippy and compares the two against 20 pinned real-world repositories. Application code is never executed for analysis — but the fibers are real, built by React's own reconciler.

source ─oxc-parser─▶ ModuleRecord ─oxc-resolver─▶ ModuleGraph ─▶ Interpreter (abstract eval over StaticValue)
      ─▶ Materializer (StaticValue ─▶ React elements; source components become proxies that call the interpreter)
      ─▶ react-dom + happy-dom ─bippy─▶ RuntimeSnapshot ──compare──▶ RuntimeSnapshot of the real app
                                                                    (happy-dom fixture, or Playwright vs a dev server)

One strategy: the interpreter understands the source, React constructs the tree. Materializer turns evaluated element values into elements of the project's own react (resolved through the module graph); every source-defined function/class component becomes a same-named proxy whose render calls Interpreter.callFunction on the source body, and memo/forwardRef/lazy/contexts/Fragment/StrictMode/Suspense/portals/host tags map to the real React APIs. Work tags, shouldSetTextContent, hoistables/singletons, memo bailouts and error boundaries are therefore React's, not a reimplementation (the earlier hand-written FiberBuilder/host-semantics mirror is gone). Proxy identity is the source closure (node + scope), so a HOC applied twice yields two component types, as at runtime. Hook cells live in a per-proxy HookFrame driven by real useState/useEffect, so setState from effects re-renders and settles before capture.

Uncertainty is materialized, not guessed: marker components $Branch/$Alternative, $Repeat, $Opaque (passed children rendered inside), $Unknown, $Text are ordinary fibers in the captured tree at the position of the uncertainty. The comparer reads the materialized tree back into a pattern (snapshotToPattern) — a marker-free tree is a plain fiber-by-fiber diff (tag, name, key, text, order); markers become try-each-alternative / absorb-any-count / slot-into-library-subtree / wildcard — and reports exact / partial / mismatch / unresolved / skipped with coverage and the first divergence path. exact is only claimed when no uncertainty was consumed.

What is modeled

  • Graph: relative/alias/tsconfig-paths/package/workspace resolution, default/named/namespace imports, export * chains, barrels, cycles, ambiguous exports. External packages are opaque unless allow-listed or modeled by a framework adapter.
  • Evaluator: lexical scopes, literals/templates/objects/spreads/members/calls/operators, raw and compiled JSX (jsx/jsxs/jsxDEV/createElement, _extends/_objectWithoutProperties, compiled classes), props.children, bounded list ops and imperative loops, hooks (useState/useReducer cells, useContext/use via nearest provider, useSyncExternalStore server snapshot, useId unknown text, effects with dep tracking), class components (fields, ctor, inheritance, this.state, setState, error boundaries), Object.assign component patterns, tagged templates, HOC composition, await in server/awaited contexts. Budgets on depth/recursion/branch fan-out/steps/element count; exhaustion is an $Unknown, never a confident truncation.
  • Frameworks (renderFramework({ framework, entry?, route? })): spa, next-app (layout/template/page/loading composition, async server components, client boundaries, Next externals; RSC mode elides server components like the Flight client), next-pages (_app), react-router (route configs, lazy routes, HydratedRouter/RouterProvider, root Layout/App, Meta/Links/ScrollRestoration, react-router-auto-routes conventions). Each has a FrameworkProfile naming only framework wrapper fibers as transparent — every name was added after observing it in a real capture, never to hide an application mismatch.

Corpus (corpus/manifest.json, results in corpus/results.json)

20 pinned repos (actual, bulletproof-react, cal-diy, epic-stack, excalidraw, graphiql, invoify, lexical, nextjs-boilerplate, nextjs-examples, puck, react-admin, react-email, react-router-templates, redux-toolkit, shadcn-ui, sonner, tanstack-query, tanstack-router, tldraw). All 20 render through the materializer with their own React (18.3 – 19.3 canary). Verified against a live dev server + Playwright:

entry framework result
react-router-templates react-router 8 exact (48 fibers + 2 text, 0 uncertainty)
sonner/test Next 14 exact (44 fibers, 0 uncertainty)
nextjs-examples/blog-starter Next 16 canary partial, coverage 100% — posts list is filesystem data at request time
tanstack-query/react/basic Vite SPA partial, coverage 100% — 100 posts from the network are an opaque slot
bulletproof-react/react-vite react-router 6 data partial, coverage 100% — Helmet's <meta> is a side effect, not a fiber

cal-diy is recorded as blocked (needs Postgres + app-store env); the rest are static-only in this environment. pnpm corpus -- --static-only replays the saved capture (.corpus/.out/<id>.capture.json) when one exists, so evaluator changes are re-checked against the same runtime tree without a dev server.

Tests / wiring

  • tests/components/*: 35 single-file component conformance fixtures (hooks/state+effects, context, class components, error boundaries, compiled classic/automatic JSX, HOCs, tagged templates, hoistables, suspending data, recursion, i18n, forms…), each rendered statically and independently mounted with react-dom; the two fiber trees must match.
  • tests/fixtures/*: 19 synthetic projects (compiled JSX, class state, portals, memo/forwardRef/lazy, re-export cycles, spread props, text normalization, opaque/derived externals, react-router data/jsx/lazy).
  • tests/framework-fixtures/*: Next App/Pages Router and react-router-auto-routes layouts.
  • Root pnpm test includes the parser vite project; root pnpm typecheck runs @bippy/parser typecheck.

packages/parser/README.md documents the pipeline, value model, materialization, comparison semantics, corpus methodology and limitations.

Link to Devin session: https://app.devin.ai/sessions/b6d14be3cb7742979b56c7d7772cb48b
Open in Devin Desktop: https://app.devin.ai/desktop/session/b6d14be3cb7742979b56c7d7772cb48b?variant=devin
Requested by: @aidenybai

devin-ai-integration Bot and others added 7 commits September 6, 2026 01:55
Co-Authored-By: Aiden Bai <aiden.bai05@gmail.com>
…ate, fixture suite

Co-Authored-By: Aiden Bai <aiden.bai05@gmail.com>
… components, native external values

Co-Authored-By: Aiden Bai <aiden.bai05@gmail.com>
Co-Authored-By: Aiden Bai <aiden.bai05@gmail.com>
…outes, derived-external uncertainty

Co-Authored-By: Aiden Bai <aiden.bai05@gmail.com>
…Meta, Links, ScrollRestoration); template matches runtime exactly

Co-Authored-By: Aiden Bai <aiden.bai05@gmail.com>
… bulletproof-react), capture replay, namespace rest/spread, README

Co-Authored-By: Aiden Bai <aiden.bai05@gmail.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

Original prompt from Aiden Bai

@aidenybai/bippy

this will be a very long march, so do not stop until we are fully done

create a package called "parser". the idea behind parser is to esentially construct Fiber from just the AST, without actually running the osurce. this will require heavy parsing of the code, liknking between modules (oxc-resolver / oxc parser). write this in TS with those packages. the idea is we can essentially render react fomr hte AST itself, or at leat a fiberish version of it whener ewe udnerstand depencies. also find you a 20 list ofcprious of real react repos to work off of , like cal.diy and such, such that you can veirfy the actua louytout form the dev server ofr compoennt renderer to the AST.. teh most important part is theo create a test harness that you can use to verify against reality. it's going to be very easy to be lazy here, we strongly want to avoid that

do extnesive reserarch first before actually going out and doing it. do it all. dont ask me quesstions. use yout best judgement, write legent code, good archietcutre adn pattenrs that are maintainable. bippy is already a very well strucutured codebase, dont fuck it up

study

@changeset-bot

changeset-bot Bot commented Sep 6, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: b7d5923

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@vercel

vercel Bot commented Sep 6, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
bippy Ready Ready Preview Sep 6, 2026 2:18pm UTC

@pkg-pr-new

pkg-pr-new Bot commented Sep 6, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/bippy@115

commit: b7d5923

Comment thread packages/parser/src/evaluate/interpreter.ts
Co-Authored-By: Aiden Bai <aiden.bai05@gmail.com>
…valuator gaps

- lowered/compiled class recognition (Babel _createClass, prototype assignment, Component.call)
- error boundary throw propagation with getDerivedStateFromError/componentDidCatch
- isolated vs outer-provider context semantics
- tagged templates, RegExp, Map/Set/Promise, reduce, scheduling globals
- optional list items for uncertain filters; structural equivalence for recursion cutoff
- callable globals (Boolean/String/Number) as array callbacks
- vite ssr anonymous default export naming normalized

Co-Authored-By: Aiden Bai <aiden.bai05@gmail.com>
…ber output

Replace the handwritten FiberBuilder/host-semantics/serializer with a
Materializer: interpreter values become React elements whose source
components are proxies calling back into the interpreter, rendered with the
project's react-dom in happy-dom and captured with bippy. Uncertainty is
kept as marker components ($Branch/$Repeat/$Opaque/$Unknown/$Text) so the
captured tree doubles as the comparison pattern. Proxies are keyed by
closure (node + scope) so HOCs applied twice yield distinct component types.
Hook state is threaded through real React state so effect-driven setState
settles before capture.

Co-Authored-By: Aiden Bai <aiden.bai05@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant