Skip to content

metro-file-map: Skip the per-file worker pass when no plugin needs it - #2240

Open
robhogan wants to merge 2 commits into
mainfrom
pr2240
Open

robhogan wants to merge 2 commits into
mainfrom
pr2240

Conversation

@robhogan

@robhogan robhogan commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

With lazy SHA-1s (Metro's default), a file only needs a worker at startup if a plugin computes data for it as it's crawled. With Metro's defaults none does, since #2220 leaves out the Haste plugin without Haste config. Even so, processBatch goes through every crawled file to find that out, checking its path for node_modules and building an array of the plugins to run on it.

This returns straight away when there's no SHA-1 to compute and every plugin's worker is lazy (lazy: true, ie it computes data on demand). It doesn't change the cache or any API.

Benchmark

This takes 5-16% off applying the crawl on a cold start (macOS: 15.4ms -> 13.0ms, Linux: 68ms -> 64ms), and 2-3% off a cold start on Linux and Windows (Linux: 297ms -> 289ms). On macOS, cache writes are 0.4ms slower. The serialise itself takes the same time, so the difference is in the rest of the write.

main #2220 This PR vs main vs #2220
macOS: Applying the crawl in a cold start 19.2ms 15.4ms 13.0ms -32.5% (-34.5% to -30.3%) -15.9% (-19.0% to -12.8%)
Linux: Applying the crawl in a cold start 83ms 68ms 64ms -23.5% (-25.3% to -20.4%) -6.5% (-8.3% to -4.1%)
Windows: Applying the crawl in a cold start 47ms 34ms 32ms -32.4% (-33.4% to -31.5%) -5.3% (-7.7% to -2.4%)
macOS: Cache write in a cold start 7.6ms 4.4ms 4.8ms -36.4% (-38.9% to -33.8%) +10.3% (+7.5% to +12.9%)
Linux: Cache write in a cold start 32ms 15.0ms 14.4ms -55.0% (-56.4% to -52.2%) -3.7% (-12.9% to +2.5%)
Windows: Cache write in a cold start 20ms 10.6ms 9.4ms -53.7% (-54.6% to -52.7%) -11.4% (-35.3% to -6.2%)
macOS: Cold start 146ms 90ms 92ms -37.1% (-43.4% to -32.7%) +1.7% (-9.6% to +13.9%)
Linux: Cold start 647ms 297ms 289ms -55.3% (-55.7% to -54.9%) -2.6% (-4.2% to -1.3%)
Windows: Cold start 358ms 162ms 158ms -55.9% (-56.5% to -55.6%) -2.6% (-4.4% to -1.7%)

Changelog

 - **[Performance]**: metro-file-map: Skip the per-file worker pass when no plugin needs it

Test plan

New unit tests. Benchmark as in #2104 (cold starts only), with 12 rounds on macOS.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 16, 2026
@robhogan
robhogan added this pull request to stack #2241 November 16, 2026 22:10
@robhogan
robhogan requested review from huntie and vzaidman November 17, 2026 15:40
@robhogan
robhogan marked this pull request as ready for review November 17, 2026 15:41
@github-actions

github-actions Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Metro review: Looks good

Risk: low · Commit: f3d5551 · 5 commands run

Since the last review, the PR keeps the early return in processBatch when there's no SHA-1 to compute and every plugin is lazy. It now builds the list of eager plugin indices once in the constructor, and both that early return and the per-file pluginsToRun filter use the list. The indices and their order match the old reduce, the tests that cover the early return still fail without it, and a paired benchmark of the eager-plugin path shows no cost, so this looks good.

No findings.

Verified
  • ❌ yarn jest --no-watchman packages/metro-file-map — 922 passed. The only failures are 13 tests in the crawler and watcher integration suites that need Watchman, which can't run in the sandbox. Every FileProcessor test passes.
  • ✅ yarn jest --no-watchman packages/metro-file-map/src/lib/__tests__/FileProcessor-test.js (with the early return in processBatch replaced by if (false)) (failed as expected) — Both cases of 'skips the per-file pass without a SHA-1 to compute' fail, so the renamed test still catches the early return being removed. Its readKeys assertion covers the mockWorkerFn assertion that was dropped. Source restored afterwards.
  • ✅ yarn typecheck
  • ✅ yarn eslint packages/metro-file-map/src/lib/FileProcessor.js packages/metro-file-map/src/lib/__tests__/FileProcessor-test.js
  • ✅ node bench/eager.js (x2) — Head and base loaded in one process, run alternately (ABAB). No significant change.

Benchmarks (base vs this PR)

Benchmark Base This PR Change
processBatch on 100k crawled files with one lazy plugin and one eager plugin whose filter rejects everything 2.31ms (2.23-3.06) 2.32ms (2.19-3.04) no significant change
Method
  • processBatch on 100k crawled files with one lazy plugin and one eager plugin whose filter rejects everything: bench/eager.js in the review directory loads FileProcessor from both checkouts in one Node process and alternates them (ABAB): 5 warm-up rounds, then 20 rounds per side. Medians with min-max range. A second run gave base 2.34 (2.17-2.91) and head 2.26 (2.12-2.96).

Not run: The description's cold-start benchmark was not repeated. The early return it measures is in the stack base commit (274335a), not in this PR's own diff, and the diff's only per-file change was benchmarked directly. Nothing in the diff is specific to Windows, so no Windows run was needed.

Automated review by Claude. Advisory only: it never approves or blocks, and a maintainer makes the call.

Base automatically changed from pr2220 to main November 19, 2026 15:03
With lazy SHA-1s (Metro's default), a file only needs a worker at startup if a plugin computes data for it as it's crawled. With Metro's defaults none does, since #2220 leaves out the Haste plugin without Haste config. Even so, `processBatch` goes through every crawled file to find that out, checking its path for `node_modules` and building an array of the plugins to run on it.

This returns straight away when there's no SHA-1 to compute and every plugin's worker is lazy (`lazy: true`, ie it computes data on demand). It doesn't change the cache or any API.

## Benchmark

This takes 5-16% off applying the crawl on a cold start (macOS: 15.4ms -> 13.0ms, Linux: 68ms -> 64ms), and 2-3% off a cold start on Linux and Windows (Linux: 297ms -> 289ms). On macOS, cache writes are 0.4ms slower. The serialise itself takes the same time, so the difference is in the rest of the write.

|  | `main` | #2220 | This PR | vs `main` | vs #2220 |
|---|---|---|---|---|---|
| macOS: Applying the crawl in a cold start | 19.2ms | 15.4ms | 13.0ms | -32.5% (-34.5% to -30.3%) | -15.9% (-19.0% to -12.8%) |
| Linux: Applying the crawl in a cold start | 83ms | 68ms | 64ms | -23.5% (-25.3% to -20.4%) | -6.5% (-8.3% to -4.1%) |
| Windows: Applying the crawl in a cold start | 47ms | 34ms | 32ms | -32.4% (-33.4% to -31.5%) | -5.3% (-7.7% to -2.4%) |
| macOS: Cache write in a cold start | 7.6ms | 4.4ms | 4.8ms | -36.4% (-38.9% to -33.8%) | +10.3% (+7.5% to +12.9%) |
| Linux: Cache write in a cold start | 32ms | 15.0ms | 14.4ms | -55.0% (-56.4% to -52.2%) | -3.7% (-12.9% to +2.5%) |
| Windows: Cache write in a cold start | 20ms | 10.6ms | 9.4ms | -53.7% (-54.6% to -52.7%) | -11.4% (-35.3% to -6.2%) |
| macOS: Cold start | 146ms | 90ms | 92ms | -37.1% (-43.4% to -32.7%) | +1.7% (-9.6% to +13.9%) |
| Linux: Cold start | 647ms | 297ms | 289ms | -55.3% (-55.7% to -54.9%) | -2.6% (-4.2% to -1.3%) |
| Windows: Cold start | 358ms | 162ms | 158ms | -55.9% (-56.5% to -55.6%) | -2.6% (-4.4% to -1.7%) |

## Changelog

```
 - **[Performance]**: metro-file-map: Skip the per-file worker pass when no plugin needs it
```

## Test plan

New unit tests. Benchmark as in #2104 (cold starts only), with 12 rounds on macOS.
@robhogan robhogan added the Auto review: Good The Metro review bot found nothing to change at this head label Nov 20, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Auto review: Good The Metro review bot found nothing to change at this head CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant