Skip to content

Add Builder.io lockfile example and verified case study - #660

Merged
sonukapoor merged 4 commits into
OWASP:mainfrom
Ayush7614:ayushbuilder
Jun 23, 2026
Merged

Add Builder.io lockfile example and verified case study#660
sonukapoor merged 4 commits into
OWASP:mainfrom
Ayush7614:ayushbuilder

Conversation

@Ayush7614

@Ayush7614 Ayush7614 commented Jun 14, 2026

Copy link
Copy Markdown
Collaborator

Document BuilderIO/builder at 1219cc4 (5,655 packages, 315 findings) with Yarn Berry + Nx monorepo lockfile coverage, yarn npm audit --all comparison, and extreme unknown-relationship triage graph.

Fixes #641

Summary

  • Adds lockfile-only snapshot examples/builder/ from BuilderIO/builder@1219cc4 (root package.json + yarn.lock)
  • Documents verified baseline scan in website/docs/case-studies/builder.mdvisual CMS / page-builder SDK monorepo coverage on one of the largest Yarn Berry lockfiles in the portfolio (5,655 packages, 315 findings)
  • 32 critical · 164 high · 100 medium · 19 low — extreme triage graph with only 2 direct findings and 313 unknown relationship rows on lockfile-only snapshot
  • Side-by-side comparison with yarn npm audit --all on the same lockfile (2 vulnerability entries vs 315 deduplicated packages)
  • Bundles Builder.io logo at website/static/img/builder-logo.png

Case-study-only PR — no shared-file edits (index.md, sidebars.ts, README.md, examples/readme.md, CHANGELOG.md, root lockfile) per CONTRIBUTING guidance.

Why this change

Builder.io (~8.7k GitHub stars) is a widely adopted visual development / headless CMS platform — drag-and-drop editing, Figma-to-code, SDKs for React, Vue, Svelte, Qwik, Angular, Next.js, and more. A committed Yarn Berry lockfile snapshot extends CVE Lite coverage into visual CMS / page-builder SDK monorepos — distinct from Mitosis (cross-framework compiler) while sharing the Builder.io ecosystem — and documents how CVE Lite behaves on a 5,655-package yarn.lock vs default yarn npm audit --all scope.

What changed

  • examples/builder/package.json + examples/builder/yarn.lock pinned to upstream revision 1219cc4
  • website/docs/case-studies/builder.md with verified scan results, yarn npm audit --all comparison, Yarn Berry unknown-relationship caveats at scale, and baseline findings table (all 32 critical + 2 direct + representative high subset; full 315 rows match scan JSON)
  • website/static/img/builder-logo.png (bundled locally from examples/next-js-builder-site/public/assets/logo.png)

Verified scan output

Parsed 5655 packages from yarn-lock (yarn.lock)
Found 315 packages (328 CVEs) with known OSV matches
Critical: 32 | High: 164 | Medium: 100 | Low: 19
Direct: 2 | Transitive: 0 | Unknown: 313
2 command groups — 2 of 315 findings with copy-and-run commands

Notable findings:

  • Only 2 direct findings: octokit@2.1.0 (high) → yarn add octokit@3.1.2 (breaking), zx@7.2.1 (medium) → yarn add zx@8.8.5 (breaking)
  • Critical sandbox/SDK cluster (unknown): @angular/ssr, @builder.io/qwik, @builder.io/qwik-city, @nguniversal/*, @remix-run/node, next, handlebars, form-data, nuxt, simple-git, vitest — framework example paths
  • Version fragmentation at scale: multiple majors of esbuild, axios, minimatch, semver, rollup, vite, svelte, lodash across examples/plugins
  • 313 skipped findings — overwhelmingly framework-sandbox packages where Yarn Berry path reconstruction is incomplete on lockfile-only MVP

yarn npm audit --all (Yarn 3.6.1, same lockfile): 2 vulnerability entries (1 high · 1 moderate) on root workspace direct deps — octokit@2.1.0 and zx@7.2.1. Case study explains full-lockfile parse vs Yarn Berry default audit scope (parallel to Storybook/Mitosis case studies).

Validation

  • npm run build
  • node dist/index.js examples/builder --verbose --all --json — 315 findings, 2 command groups, 5,655 packages parsed
  • yarn npm audit --all run from examples/builder/ — 2 entries documented in case study
  • Case study numbers match live scan JSON (cve-lite-scan-2026-06-14T22-44-44.json)
  • Baseline table: all 32 critical + 2 direct + representative high subset (315-row full list in JSON)
  • Docusaurus site builds (CI)

User-facing impact

Does this change:

  • affect scanning behavior
  • affect output formatting
  • affect JSON output
  • affect docs only

Notes

Issue #641 preliminary scan (v1.22.0, 2026-06-12) numbers match the verified live scan on 2026-06-14: 315 findings, 2/0/313 relationship split, 2 fix groups, 2/315 first-pass coverage. Baseline only — no fake "after" remediation results.

cc: @sonukapoor

@sonukapoor sonukapoor left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good foundation here - the fixture is solid and the baseline numbers check out. A few things to fix:

Missing final section - the doc ends at ## Baseline findings but every case study needs a ## Want your project reviewed? section as the last section. You can copy it from any of the existing studies (analog.md, nestjs.md etc.).

Extra sections - ## Lockfile scope and ## Scan verification aren't in the required template. The lockfile scope content fits naturally inside ## What this case study demonstrates, and the scan verification content can go inside ## Scan command. Could you fold them in to keep the structure consistent across all studies?

Once those are sorted this is good to go.

@Ayush7614

Copy link
Copy Markdown
Collaborator Author

Thanks for the review, @sonukapoor — addressed the template feedback in ba8a6df:

  • Folded Lockfile scope into What this case study demonstrates
  • Merged Scan verification into Scan command (verification table + reproduce commands)
  • Added Want your project reviewed? at the end (copied from analog.md)

Ready for another look when you have a moment.

@sonukapoor sonukapoor left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Three things to fix:

The "Want your project reviewed?" closing section links to github.com/sonukapoor/cve-lite-cli/issues - please change this to github.com/OWASP/cve-lite-cli/issues.

The logo is a PNG - all other case study logos in website/static/img/ are SVGs. Please swap for an SVG export from Builder.io's brand assets.

The next fix hints in the critical findings table show version downgrades (e.g. next@15.x to 12.3.5). This reflects the minimum patched version in the OSV advisory data and can confuse readers. Please add a short note near that table explaining that fix hints reflect the earliest patched release in the advisory - developers should verify against the Next.js security page before upgrading.

Branch is also behind main - please rebase with git fetch origin && git rebase origin/main && git push --force-with-lease.

@Ayush7614

Copy link
Copy Markdown
Collaborator Author

Thanks @sonukapoor — all three points addressed and rebased onto latest main (ahead 3 / behind 0):

  1. Link fixed — "Want your project reviewed?" now points to https://github.com/OWASP/cve-lite-cli/issues (was sonukapoor fork).

  2. Logo swapped to SVG — replaced builder-logo.png with builder-logo.svg from Builder.io brand assets (matches the other case study logos in website/static/img/).

  3. Fix-hint note added — added a short callout above the critical findings table explaining that fix hints reflect the earliest patched release in OSV advisory data (e.g. next@15.x12.3.5 is not a downgrade recommendation) and that developers should verify against the upstream security page before upgrading.

Ready for another look.

Ayush7614 added a commit to Ayush7614/cve-lite-cli that referenced this pull request Jun 19, 2026
- Point "Want your project reviewed?" link to OWASP/cve-lite-cli/issues
- Replace builder-logo.png with official brand SVG (matches other case studies)
- Add note explaining OSV fix hints can show earliest patched version (next downgrade)

Co-authored-by: Cursor <cursoragent@cursor.com>
Ayush7614 added a commit to Ayush7614/cve-lite-cli that referenced this pull request Jun 19, 2026
- Point "Want your project reviewed?" link to OWASP/cve-lite-cli/issues
- Replace builder-logo.png with official brand SVG (matches other case studies)
- Add note explaining OSV fix hints can show earliest patched version (next downgrade)
Ayush7614 added a commit to Ayush7614/cve-lite-cli that referenced this pull request Jun 21, 2026
- Point "Want your project reviewed?" link to OWASP/cve-lite-cli/issues
- Replace builder-logo.png with official brand SVG (matches other case studies)
- Add note explaining OSV fix hints can show earliest patched version (next downgrade)
@Ayush7614

Copy link
Copy Markdown
Collaborator Author

Thanks @sonukapoor — rebased onto latest main in 8912e4c (now includes merged #656, #657, and #658). Branch is up to date and ready for approval.

@sonukapoor sonukapoor left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 2-vs-315 contrast is a compelling headline and the fixture is legitimately interesting. Three things to address before merge.

The Before vs After table having only a baseline row is a structural issue - the project rule requires at least one measured pass. The justification (both fix commands are breaking majors that weren't applied) is understandable, but other reviewers will flag it too. One option: run the two commands against a branch, rescan, record the row, then restore with git checkout -- .. Even if the count drops by only 2, the table shows the process worked.

Comment thread website/docs/case-studies/builder.md Outdated
@@ -0,0 +1,263 @@
# Builder.io Case Study

> Verified baseline scan — CVE Lite CLI v1.22.0 · 2026-06-14

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Em dashes (—) are not allowed anywhere in project docs - please replace every instance with a hyphen or rewrite the sentence. There are 15+ throughout this file. This one is the first, but they appear on lines 3, 11, 20, 26, 28, 34, 38, 62, 64, 66, 78, 91, 111, 121, 149, 167, and 249.

Comment thread website/docs/case-studies/builder.md Outdated

---

## Remaining risk

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The required heading format includes "after": ## Remaining risk after baseline scan. Can you update?

Ayush7614 and others added 3 commits June 24, 2026 02:15
Fold Lockfile scope into What this case study demonstrates, merge Scan verification into Scan command, and add Want your project reviewed section per PR review.
- Point "Want your project reviewed?" link to OWASP/cve-lite-cli/issues
- Replace builder-logo.png with official brand SVG (matches other case studies)
- Add note explaining OSV fix hints can show earliest patched version (next downgrade)
Ayush7614 added a commit to Ayush7614/cve-lite-cli that referenced this pull request Jun 23, 2026
Record two-pass fix results (336→332→331), remove em dashes, and rename the remaining-risk heading per PR OWASP#660 review.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Ayush7614

Copy link
Copy Markdown
Collaborator Author

Thanks @sonukapoor — all three review items addressed in d38d846:

  1. Before vs After table — applied yarn add octokit@3.1.2 and yarn add zx@8.8.5 one at a time on a test checkout of BuilderIO/builder@1219cc4, rescanned through the committed examples/builder lockfile fixture after each pass (CVE Lite v1.25.0 · 2026-06-22). Findings 336 → 332 → 331; both direct levers cleared.
  2. Em dashes — replaced every em dash in builder.md with a hyphen or rewritten sentence.
  3. Remaining risk heading — renamed to ## Remaining risk after baseline scan.

Rebased onto latest main. Ready for another look.

Record two-pass fix results (336→332→331), remove em dashes, and rename the remaining-risk heading per PR OWASP#660 review.

@sonukapoor sonukapoor left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the previous issues are resolved - measured passes are in, SVG logo, correct OWASP URL, and the heading format is right. The Before vs After story (336 to 331 across two breaking majors) is a good one. Ready to merge.

@sonukapoor
sonukapoor merged commit 5dff265 into OWASP:main Jun 23, 2026
9 checks passed
@sonukapoor

Copy link
Copy Markdown
Collaborator

Merged - thank you @Ayush7614!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Builder.io lockfile example and verified case study

2 participants