Skip to content

Resolver: Record what failed resolutions observed, on UnableToResolveError - #2137

Draft
robhogan wants to merge 4 commits into
robhogan/plugin-change-summariesfrom
robhogan/observe-failed-resolutions
Draft

robhogan wants to merge 4 commits into
robhogan/plugin-change-summariesfrom
robhogan/observe-failed-resolutions

Conversation

@robhogan

Copy link
Copy Markdown
Collaborator

#1949 records what a successful resolution observes of the file system, but a resolution that fails throws and its record is discarded. A failed import depends on the file system just as much - creating the missing file, or a package.json that redirects to it, can make it resolve - so anything that wants to know when to retry needs the same record.

This attaches it to UnableToResolveError as unstable_observations. With unstable_incrementalResolution on, it's the record from the failed attempt. With it off, Metro replays the failure once with a fresh record. That's sound because resolution is synchronous and observes the file system only through the capabilities bound to the record, so the replay sees the same state and fails the same way. It costs one extra resolution per failure, and only for failures that break the build.

Optional dependencies aren't replayed, since their failures are expected and they re-resolve on every traversal. If a replay succeeds, a custom resolver is reading something outside its resolution context. Metro warns about that and throws the original error, without observations.

Within Metro, a hard failure already retries on the next change because DeltaCalculator keeps the module pending, so nothing here reads the record yet. It's a draft to settle the shape of the API.

Changelog: Internal

Test plan:
New unit tests.

robhogan and others added 4 commits October 30, 2026 15:31
…ble_incrementalResolution

Stacked on #1948.

The first consumer of the `Observations` API from #1944. When `resolver.unstable_incrementalResolution` is enabled, `ModuleResolver.resolveDependency` creates an observations record at the start of each resolution, and everything that resolution looks up in the file map is recorded into it. The record is returned on the `BundlerResolution` as `unstable_observations`. Nothing reads it yet - invalidating from it is the next step - so there's no behaviour change, and with the flag off nothing is allocated.

The resolution context given to `metro-resolver` doesn't change. `ModuleResolver` already builds a context per resolution, so when the flag is on it binds the file system capabilities to that resolution's record (`fileSystemLookup: p => fileSystemLookup(p, observations)`), and when it's off it passes the same functions as before. That means a custom `resolveRequest` records whatever it looks up through the context, without knowing about any of this. The observations parameter only appears on Metro's internal option types.

What's recorded:
 - `fileSystemLookup`, and `resolveAsset` through it.
 - `getPackageForModule`, down to `hierarchicalLookup`. There's no memo in the way since #1948.
 - `doesFileExist`. `FileSystem.exists` can't record, so when recording this is answered by `lookup`, which agrees with it - true for exactly the paths found to be a file.

Putting the record on the `BundlerResolution` means `DependencyGraph`'s whole-resolution cache replays it for free - every origin given a cached resolution gets its observations too. The one other memo is the empty module, which is resolved once and cached, so a resolution that lands on it returns a copy with the union of what led there and what resolving the empty module observed.

Not included, to follow:
 - The contents of `package.json`. `getPackage` reads with `fs` outside the file map, and Metro has no way to turn an absolute path into a canonical one. The file is recorded in `existence` by `hierarchicalLookup`, so adding or removing one is covered, but editing one isn't.
 - Haste lookups, which have no bucket yet.
 - Failed resolutions. They throw, so there's nowhere to put what they observed.

Resolution time in isolation, replaying every `DependencyGraph.resolveDependency` call made by one iOS dev build of the benchmark app (described in the test plan), through the whole-resolution cache. Nothing is transformed or serialised, so these are percentages of resolution, not of a build. 7 interleaved rounds. Resolutions are identical with the flag on and off.

| scenario | #1948 | this diff, flag off | this diff, flag on |
|---|---|---|---|
| first build (ms) | 95.9 | 95.8 (−0.1%) | 111.6 (+16.5%) |
| rebuild, nothing changed (ms) | 7.61 | 7.64 (+0.4%) | 7.80 (+2.1%) |
| after a source edit, re-resolving the edited module's dependencies (µs) | 153 | 159 (+4.1%) | 180 (+13.2%) |
| after a `package.json` change, re-resolving everything (ms) | 79.8 | 80.1 (+0.4%) | 93.1 (+16.2%) |
| heap retained after resolving (MB) | | 5.3 | 18.8 (3.5x) |

Flag off is within noise of #1948. (The source edit row is 8 resolutions, so that's under a microsecond each, and the runs overlap.)

Flag on costs about 16% of resolution time wherever everything is resolved, which is ~2µs per resolution, and 3.5x the retained heap. On the benchmark app that's 100k entries across 6.9k distinct resolutions - a median of 14 each, 90th percentile 27 - naming 28k distinct paths. Nearly all of it is `existence`, and most of that is extension and platform candidates that missed (`a.ios.js`, `a.native.js`, ...), which can't collapse because the directory they're in exists. That's the memory to go after before this is worth turning on, e.g. by recording the directory once instead.

```
yarn jest packages/metro-file-map packages/metro-resolver packages/metro/src/node-haste packages/metro/src/DeltaBundler packages/metro/src/integration_tests
yarn flow check
yarn typecheck-ts
yarn verify-api-snapshots
```
New tests in `resolver-test.js` assert the exact sets recorded for a relative import's candidates, each level probed for a package, a symlink (recorded as `content`, with lookups recorded by real path), asset variants, and a resolution that lands on the empty module. They also check that there's no `unstable_observations` by default, and that origins sharing a cached resolution share its observations.

**Benchmark app** - a synthetic mid-size RN 0.87 app: 23 common dependencies (Reanimated, React Navigation, TanStack Query, lodash, etc.) and ~1,000 generated first-party modules, 60% of them in a workspace package outside `projectRoot` with its own `node_modules`. The iOS dev bundle is 2.8k modules, from a file map of 34k files. Timings replay the `resolveDependency` calls a real build of it makes against a fresh `DependencyGraph` per process, in interleaved rounds, with source and `package.json` edits made on disk and picked up by the watcher. A real build of this app takes around 4s even with a warm transform cache, so resolution is about 2% of it.

Changelog: Internal
Stacked on #1949.

## Summary
#1949 records what a resolution looks up in the file map. It doesn't record Haste lookups, which aren't paths - the resolver asks the Haste map for a module or package by name, for every bare specifier when `allowHaste` is set. A module of that name appearing later changes the result, so the name needs observing whether or not it was found.

This adds `haste` to `ResolutionObservations`, the set of Haste names a resolution looked up. They're recorded in the two Haste closures that `ModuleResolver` already creates per resolution, so nothing else changes and the resolution context is untouched. It's the bare name that's recorded, not the platform - `HastePlugin` falls back from the platform to `native` and then to the generic module of that name, so a change to any of those bindings can change the answer. Haste packages share the bucket, since they share the namespace.

`metro-file-map` doesn't change. Its `Observations` is an interface, and Metro's record is a wider type that satisfies it, which is why that type wasn't exported.

Two things keep this cheap:
 - Nothing is observed when no file can be given a Haste name - no `hasteImplModulePath` and `enableGlobalPackages: false`, which is Metro's default. The Haste map is empty then and always will be, so every lookup is a guaranteed miss, and recording one per bare specifier would be paying to observe something that can't change. `createModuleResolver` works this out from config, by the same rule `HastePlugin`'s filter uses.
 - The set is created on the first name recorded, and `haste` is `null` until then. Most resolutions are relative imports that never consult Haste. Allocating a set for each when Haste is enabled cost ~0.9MB on the benchmark app for ~0.2MB of entries.

Nothing reads these yet. Invalidating from them needs `HastePlugin` to report which names changed binding in a batch - it's the only thing that knows, since a removed file's name is gone from the map by the time anyone else sees the change event. That's to follow, alongside invalidation generally.

The empty module is resolved with `allowHaste: false`, so it observes no names and there's nothing to merge for it.

## Benchmark
Resolution time in isolation, flag on in both columns, through the whole-resolution cache (see the test plan). No measurable cost either way - everything is within about half a percent, inside the run-to-run spread.

| scenario | #1949 | this diff | #1949, Haste enabled | this diff, Haste enabled |
|---|---|---|---|---|
| first build (ms) | 112.4 | 111.8 | 111.2 | 110.6 |
| rebuild, nothing changed (ms) | 7.75 | 7.78 | 7.81 | 7.79 |
| after a source edit, the edited module's dependencies (µs) | 177 | 168 | 170 | 172 |
| after a `package.json` change, re-resolving everything (ms) | 93.2 | 93.2 | 93.6 | 94.1 |
| heap retained after resolving (MB) | 18.8 | 18.8 | | 19.1 |

With Haste enabled, 27% of distinct resolutions look up a name (1.9k of 6.9k), one name each, over 72 distinct names. With it inert, no resolution carries a set.

## Test plan
```
yarn jest packages/metro-file-map packages/metro-resolver packages/metro/src/node-haste packages/metro/src/DeltaBundler packages/metro/src/integration_tests
yarn flow check
yarn typecheck-ts
yarn verify-api-snapshots
```
New tests in `resolver-test.js`: nothing is observed when no file can be given a Haste name, a name is recorded when it's found and when it isn't, a deep import records only its package name, a relative import records nothing, and a global package name is recorded. The existing exact-set expectations gain `haste: null`.

**Benchmark app** - a synthetic mid-size RN 0.87 app: 23 common dependencies (Reanimated, React Navigation, TanStack Query, lodash, etc.) and ~1,000 generated first-party modules, 60% of them in a workspace package outside `projectRoot` with its own `node_modules`. The iOS dev bundle is 2.8k modules, from a file map of 34k files. Timings replay the `resolveDependency` calls a real build of it makes against a fresh `DependencyGraph` per process, in interleaved rounds, with source and `package.json` edits made on disk and picked up by the watcher. "Haste enabled" is the same app with `enableGlobalPackages: true`, which gives its first-party packages Haste names. These timings isolate resolution - no transformation, serialisation or file reads are included. A real build of this app takes around 4s even with a warm transform cache, so resolution is about 2% of it.

Changelog: Internal
…ange event

Stacked on #1952.

## Summary
A file map plugin often knows things about a batch of changes that nothing else can work out. The motivating case is Haste - #1952 records the Haste names a resolution looks up, and to invalidate from those a consumer needs to know which names were rebound by a change. It can't derive that from the `ChangeEvent`, which carries paths and modified times, and by the time it sees the event a removed file's name is already gone from the map. `HastePlugin` has it to hand - its `onChanged` is given each removed file's old name and each added file's new one.

This gives plugins a way to say so. `FileMapPlugin.onChanged` may now return a summary, and the file map publishes what each plugin returned as `ChangeEvent.pluginChanges`, by plugin name. A plugin that returns nothing is absent, so existing plugins are unaffected - the summary is a third type parameter on `FileMapPlugin` that defaults to `void`. `getPluginChanges(event, plugin)` reads an entry with the plugin's own summary type, so a consumer never writes the name or casts.

The summary describes exactly the batch in `changes`. Plugins are already updated synchronously, immediately before the emit, so there's no ordering for a consumer to get wrong - which is the problem with the alternative of asking the plugin afterwards. There are two emit sites: the batched watch path updates plugins inline, and a recrawl updates them inside `#applyFileDelta`, which now returns what they reported along with the changes. The initial build goes through `#applyFileDelta` too but emits nothing, so its summaries go unused.

`HastePlugin` is the first user. It reports `changedNames`, the names bound or unbound for any platform by the files added and removed - a bare name, matching what #1952 records.

Plugin names now have to be unique across all of a file map's plugins. That was only checked for plugins with a worker, but persisted plugin state is keyed by name for every plugin, so two that shared a name would already have overwritten each other's state.

Two things worth knowing:
 - On a cold start `onChanged` sees every file as added, so `HastePlugin` builds a set of every name for a summary nobody reads. That's nothing where Haste is inert, which is Metro's default, and transient otherwise. If it matters, the alternative is an optional method the file map calls only for a batch it's about to emit.
 - A modified file reaches plugins in `modifiedFiles` with its new data only, and `HastePlugin.onChanged` doesn't read that list. So an edit that changes a file's Haste name (a docblock, or a global package's `name`) is neither applied to the map nor reported here. That predates this, and can't happen with a path-based `hasteImpl`. `FileSystemChangeAggregator` keeps each file's pre-batch metadata, so giving plugins the old value is feasible, separately.

Nothing consumes this yet.

## Test plan
```
yarn jest packages/metro-file-map packages/metro-resolver packages/metro/src/node-haste packages/metro/src/DeltaBundler packages/metro/src/integration_tests packages/metro/src/__tests__ packages/metro/src/Server
yarn flow check
yarn typecheck-ts
yarn verify-api-snapshots
```
 - `HastePlugin-test.js`: `onChanged` reports the names bound and unbound, and nothing when no file with a name was added or removed.
 - `index-test.js`: in watch mode an added and a removed file put both names on the event, read back through `getPluginChanges`, and plugins with nothing to say are absent. A recrawl publishes what plugins reported too. Two plugins sharing a name throw, with or without a worker.

Changelog: Internal
…Error

#1949 records what a successful resolution observes of the file system, but a resolution that fails throws and its record is discarded. A failed import depends on the file system just as much - creating the missing file, or a `package.json` that redirects to it, can make it resolve - so anything that wants to know when to retry needs the same record.

This attaches it to `UnableToResolveError` as `unstable_observations`. With `unstable_incrementalResolution` on, it's the record from the failed attempt. With it off, Metro replays the failure once with a fresh record. That's sound because resolution is synchronous and observes the file system only through the capabilities bound to the record, so the replay sees the same state and fails the same way. It costs one extra resolution per failure, and only for failures that break the build.

Optional dependencies aren't replayed, since their failures are expected and they re-resolve on every traversal. If a replay succeeds, a custom resolver is reading something outside its resolution context. Metro warns about that and throws the original error, without observations.

Within Metro, a hard failure already retries on the next change because `DeltaCalculator` keeps the module pending, so nothing here reads the record yet. It's a draft to settle the shape of the API.

Changelog: Internal

Test plan:
New unit tests.
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 30, 2026
@robhogan
robhogan force-pushed the robhogan/plugin-change-summaries branch 2 times, most recently from 7866a3a to 08303d7 Compare November 18, 2026 13:05

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant