Skip to content

Commit 9c41cb8

Browse files
committed
Resolver: Observe Haste names under unstable_incrementalResolution
Stacked on #1949. ## Summary #1949 records what a resolution looks up in the file map. It doesn't record Haste lookups, which aren't paths - the resolver asks the Haste map for a module or package by name, for every bare specifier when `allowHaste` is set. A module of that name appearing later changes the result, so the name needs observing whether or not it was found. This adds `haste` to `ResolutionObservations`, the set of Haste names a resolution looked up. They're recorded in the two Haste closures that `ModuleResolver` already creates per resolution, so nothing else changes and the resolution context is untouched. It's the bare name that's recorded, not the platform - `HastePlugin` falls back from the platform to `native` and then to the generic module of that name, so a change to any of those bindings can change the answer. Haste packages share the bucket, since they share the namespace. `metro-file-map` doesn't change. Its `Observations` is an interface, and Metro's record is a wider type that satisfies it, which is why that type wasn't exported. Two things keep this cheap: - Nothing is observed when no file can be given a Haste name - no `hasteImplModulePath` and `enableGlobalPackages: false`, which is Metro's default. The Haste map is empty then and always will be, so every lookup is a guaranteed miss, and recording one per bare specifier would be paying to observe something that can't change. `createModuleResolver` works this out from config, by the same rule `HastePlugin`'s filter uses. - The set is created on the first name recorded, and `haste` is `null` until then. Most resolutions are relative imports that never consult Haste. Allocating a set for each when Haste is enabled cost ~0.9MB on the benchmark app for ~0.2MB of entries. Nothing reads these yet. Invalidating from them needs `HastePlugin` to report which names changed binding in a batch - it's the only thing that knows, since a removed file's name is gone from the map by the time anyone else sees the change event. That's to follow, alongside invalidation generally. The empty module is resolved with `allowHaste: false`, so it observes no names and there's nothing to merge for it. ## Benchmark Resolution time in isolation, flag on in both columns, through the whole-resolution cache (see the test plan). No measurable cost either way - everything is within about half a percent, inside the run-to-run spread. | scenario | #1949 | this diff | #1949, Haste enabled | this diff, Haste enabled | |---|---|---|---|---| | first build (ms) | 112.4 | 111.8 | 111.2 | 110.6 | | rebuild, nothing changed (ms) | 7.75 | 7.78 | 7.81 | 7.79 | | after a source edit, the edited module's dependencies (µs) | 177 | 168 | 170 | 172 | | after a `package.json` change, re-resolving everything (ms) | 93.2 | 93.2 | 93.6 | 94.1 | | heap retained after resolving (MB) | 18.8 | 18.8 | | 19.1 | With Haste enabled, 27% of distinct resolutions look up a name (1.9k of 6.9k), one name each, over 72 distinct names. With it inert, no resolution carries a set. ## Test plan ``` yarn jest packages/metro-file-map packages/metro-resolver packages/metro/src/node-haste packages/metro/src/DeltaBundler packages/metro/src/integration_tests yarn flow check yarn typecheck-ts yarn verify-api-snapshots ``` New tests in `resolver-test.js`: nothing is observed when no file can be given a Haste name, a name is recorded when it's found and when it isn't, a deep import records only its package name, a relative import records nothing, and a global package name is recorded. The existing exact-set expectations gain `haste: null`. **Benchmark app** - a synthetic mid-size RN 0.87 app: 23 common dependencies (Reanimated, React Navigation, TanStack Query, lodash, etc.) and ~1,000 generated first-party modules, 60% of them in a workspace package outside `projectRoot` with its own `node_modules`. The iOS dev bundle is 2.8k modules, from a file map of 34k files. Timings replay the `resolveDependency` calls a real build of it makes against a fresh `DependencyGraph` per process, in interleaved rounds, with source and `package.json` edits made on disk and picked up by the watcher. "Haste enabled" is the same app with `enableGlobalPackages: true`, which gives its first-party packages Haste names. These timings isolate resolution - no transformation, serialisation or file reads are included. A real build of this app takes around 4s even with a warm transform cache, so resolution is about 2% of it. Changelog: Internal
1 parent d0ced09 commit 9c41cb8

4 files changed

Lines changed: 154 additions & 11 deletions

File tree

‎packages/metro/src/DeltaBundler/__tests__/resolver-test.js‎

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3041,6 +3041,7 @@ function dep(name: string): TransformResultDependency {
30413041
'package.json',
30423042
),
30433043
content: new Set(),
3044+
haste: null,
30443045
},
30453046
});
30463047
});
@@ -3087,6 +3088,7 @@ function dep(name: string): TransformResultDependency {
30873088
'node_modules/pkg/main.js',
30883089
),
30893090
content: new Set(),
3091+
haste: null,
30903092
},
30913093
});
30923094
});
@@ -3109,6 +3111,7 @@ function dep(name: string): TransformResultDependency {
31093111
),
31103112
// Retargeting the link changes what every path through it means
31113113
content: canonical('link'),
3114+
haste: null,
31123115
},
31133116
});
31143117
});
@@ -3134,6 +3137,7 @@ function dep(name: string): TransformResultDependency {
31343137
'package.json',
31353138
),
31363139
content: new Set(),
3140+
haste: null,
31373141
});
31383142
});
31393143

@@ -3173,6 +3177,7 @@ function dep(name: string): TransformResultDependency {
31733177
'package.json',
31743178
),
31753179
content: new Set(),
3180+
haste: null,
31763181
},
31773182
});
31783183
});
@@ -3185,6 +3190,110 @@ function dep(name: string): TransformResultDependency {
31853190
expect(second).toBe(first);
31863191
expect(second.unstable_observations?.existence.size).toBeGreaterThan(0);
31873192
});
3193+
describe('Haste names', () => {
3194+
const withHaste: InputConfigT = {
3195+
resolver: {
3196+
hasteImplModulePath: path.join(
3197+
__dirname,
3198+
'../__fixtures__/hasteImpl.js',
3199+
),
3200+
unstable_incrementalResolution: true,
3201+
},
3202+
};
3203+
3204+
test('are not observed when no file can be given one', async () => {
3205+
setMockFileSystem({
3206+
'index.js': '',
3207+
node_modules: {pkg: {'index.js': ''}},
3208+
});
3209+
resolver = await createResolver(incremental, 'ios');
3210+
// A bare specifier is looked up in the Haste map, which is empty and
3211+
// will stay that way
3212+
expect(
3213+
resolver.resolve(p('/root/index.js'), dep('pkg'))
3214+
.unstable_observations?.haste,
3215+
).toBe(null);
3216+
});
3217+
3218+
test('records a name that is found', async () => {
3219+
setMockFileSystem({
3220+
'index.js': '',
3221+
'hasteModule.js': '@providesModule hasteModule',
3222+
});
3223+
resolver = await createResolver(withHaste, 'ios');
3224+
const resolution = resolver.resolve(
3225+
p('/root/index.js'),
3226+
dep('hasteModule'),
3227+
);
3228+
expect(resolution.filePath).toBe(p('/root/hasteModule.js'));
3229+
expect(resolution.unstable_observations?.haste).toEqual(
3230+
new Set(['hasteModule']),
3231+
);
3232+
});
3233+
3234+
test('records a name that is not found, since it may appear later', async () => {
3235+
setMockFileSystem({
3236+
'index.js': '',
3237+
node_modules: {pkg: {'index.js': ''}},
3238+
});
3239+
resolver = await createResolver(withHaste, 'ios');
3240+
const resolution = resolver.resolve(p('/root/index.js'), dep('pkg'));
3241+
expect(resolution.filePath).toBe(
3242+
p('/root/node_modules/pkg/index.js'),
3243+
);
3244+
expect(resolution.unstable_observations?.haste).toEqual(
3245+
new Set(['pkg']),
3246+
);
3247+
});
3248+
3249+
test('records only the package name of a deep import', async () => {
3250+
setMockFileSystem({
3251+
'index.js': '',
3252+
node_modules: {pkg: {lib: {'thing.js': ''}}},
3253+
});
3254+
resolver = await createResolver(withHaste, 'ios');
3255+
expect(
3256+
resolver.resolve(p('/root/index.js'), dep('pkg/lib/thing'))
3257+
.unstable_observations?.haste,
3258+
).toEqual(new Set(['pkg']));
3259+
});
3260+
3261+
test('records nothing for a relative import, which does not consult Haste', async () => {
3262+
setMockFileSystem({'index.js': '', 'a.js': ''});
3263+
resolver = await createResolver(withHaste, 'ios');
3264+
expect(
3265+
resolver.resolve(p('/root/index.js'), dep('./a'))
3266+
.unstable_observations?.haste,
3267+
).toBe(null);
3268+
});
3269+
3270+
test('records a global package name', async () => {
3271+
setMockFileSystem({
3272+
'index.js': '',
3273+
aPackage: {
3274+
'package.json': JSON.stringify({name: 'aPackage'}),
3275+
'index.js': '',
3276+
},
3277+
});
3278+
resolver = await createResolver(
3279+
{
3280+
resolver: {
3281+
enableGlobalPackages: true,
3282+
unstable_incrementalResolution: true,
3283+
},
3284+
},
3285+
'ios',
3286+
);
3287+
const resolution = resolver.resolve(
3288+
p('/root/index.js'),
3289+
dep('aPackage'),
3290+
);
3291+
expect(resolution.filePath).toBe(p('/root/aPackage/index.js'));
3292+
expect(resolution.unstable_observations?.haste).toEqual(
3293+
new Set(['aPackage']),
3294+
);
3295+
});
3296+
});
31883297
});
31893298

31903299
describe('schemeResolvers', () => {

‎packages/metro/src/DeltaBundler/types.js‎

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -162,12 +162,19 @@ export type AllowOptionalDependenciesWithOptions = {
162162
export type AllowOptionalDependencies =
163163
boolean | AllowOptionalDependenciesWithOptions;
164164

165-
// What a resolution observed of the file system: canonical paths whose
166-
// addition or removal can change its result, and those whose modification
167-
// can too. Satisfies `metro-file-map`'s `Observations`, which populates it.
165+
// What a resolution observed. `existence` and `content` are canonical paths
166+
// whose addition or removal can change its result, and those whose
167+
// modification can too. They satisfy `metro-file-map`'s `Observations`, which
168+
// populates them.
169+
//
170+
// `haste` is every Haste module or package name the resolution looked up,
171+
// whether or not it was found, since one appearing later changes the result.
172+
// It is null if there were none, which is always the case when no file can be
173+
// given a Haste name - nothing is paid to observe what cannot change.
168174
export type ResolutionObservations = Readonly<{
169175
existence: Set<string>,
170176
content: Set<string>,
177+
haste: ?Set<string>,
171178
}>;
172179

173180
export type BundlerResolution = Readonly<{

‎packages/metro/src/node-haste/DependencyGraph/ModuleResolution.js‎

Lines changed: 30 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,9 @@ type Options = Readonly<{
4848
) => ReturnType<FileSystemLookup>,
4949
getHasteModulePath: (name: string, platform: ?string) => ?string,
5050
getHastePackagePath: (name: string, platform: ?string) => ?string,
51+
// Whether any file can be given a Haste name. If not, Haste lookups always
52+
// miss, and are not worth observing.
53+
isHasteEnabled: boolean,
5154
mainFields: ReadonlyArray<string>,
5255
getPackage: (packageJsonPath: string) => ?PackageJson,
5356
getPackageForModule: (
@@ -77,8 +80,8 @@ type Options = Readonly<{
7780

7881
// Every record is created here, with the same properties in the same order,
7982
// so that reading them stays monomorphic on the lookup hot path.
80-
function createObservations(): ResolutionObservations {
81-
return {existence: new Set(), content: new Set()};
83+
function createObservations(): {...ResolutionObservations} {
84+
return {existence: new Set(), content: new Set(), haste: null};
8285
}
8386

8487
export class ModuleResolver {
@@ -213,7 +216,7 @@ export class ModuleResolver {
213216
allowHaste: boolean,
214217
platform: string | null,
215218
resolverOptions: ResolverInputOptions,
216-
observations: ?ResolutionObservations,
219+
observations: ?{...ResolutionObservations},
217220
): Resolution {
218221
const {
219222
assetExts,
@@ -235,6 +238,18 @@ export class ModuleResolver {
235238
unstable_enablePackageExports,
236239
unstable_incrementalResolution,
237240
} = this._options;
241+
// Most resolutions never consult Haste, so the set is created on demand.
242+
const observeHasteName =
243+
observations != null && this._options.isHasteEnabled
244+
? (name: string) => {
245+
let names = observations.haste;
246+
if (names == null) {
247+
names = new Set<string>();
248+
observations.haste = names;
249+
}
250+
names.add(name);
251+
}
252+
: null;
238253

239254
return Resolver.resolve(
240255
createDefaultContext(
@@ -268,10 +283,17 @@ export class ModuleResolver {
268283
? resolveAsset
269284
: (dirPath, assetName, extension) =>
270285
resolveAsset(dirPath, assetName, extension, observations),
271-
resolveHasteModule: (name: string) =>
272-
this._options.getHasteModulePath(name, platform),
273-
resolveHastePackage: (name: string) =>
274-
this._options.getHastePackagePath(name, platform),
286+
// A name is recorded whether or not it is found, and for any
287+
// platform, since `HastePlugin` falls back from the platform to
288+
// `native` and then to the generic module of that name.
289+
resolveHasteModule: (name: string) => {
290+
observeHasteName?.(name);
291+
return this._options.getHasteModulePath(name, platform);
292+
},
293+
resolveHastePackage: (name: string) => {
294+
observeHasteName?.(name);
295+
return this._options.getHastePackagePath(name, platform);
296+
},
275297
resolveRequest,
276298
schemeResolvers,
277299
sourceExts,
@@ -323,7 +345,7 @@ export class ModuleResolver {
323345
}
324346
// The empty module is resolved once and cached, so a resolution that
325347
// lands on it depends on what that resolution observed as well as on
326-
// what led here.
348+
// what led here. It is resolved without Haste, so observes no names.
327349
const emptyObservations = emptyModule.unstable_observations;
328350
if (emptyObservations != null) {
329351
for (const canonicalPath of emptyObservations.existence) {

‎packages/metro/src/node-haste/DependencyGraph/createModuleResolver.js‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,11 @@ export default function createModuleResolver({
6565
emptyModulePath: config.resolver.emptyModulePath,
6666
extraNodeModules: config.resolver.extraNodeModules,
6767
fileSystemLookup,
68+
// As `HastePlugin` decides which files to give a name: packages only if
69+
// global packages are enabled, and other files only with a `hasteImpl`.
70+
isHasteEnabled:
71+
config.resolver.enableGlobalPackages ||
72+
config.resolver.hasteImplModulePath != null,
6873
getHasteModulePath: (name, platform) =>
6974
hasteMap.getModule(name, platform, true),
7075
getHastePackagePath: (name, platform) =>

0 commit comments

Comments
 (0)