Skip to content

Commit f4682e7

Browse files
committed
Resolver: Observe Haste names under unstable_incrementalResolution
Stacked on #1949. ## Summary #1949 records what a resolution looks up in the file map. It doesn't record Haste lookups, which aren't paths - the resolver asks the Haste map for a module or package by name, for every bare specifier when `allowHaste` is set. A module of that name appearing later changes the result, so the name needs observing whether or not it was found. This adds `haste` to `ResolutionObservations`, the set of Haste names a resolution looked up. They're recorded in the two Haste closures that `ModuleResolver` already creates per resolution, so nothing else changes and the resolution context is untouched. It's the bare name that's recorded, not the platform - `HastePlugin` falls back from the platform to `native` and then to the generic module of that name, so a change to any of those bindings can change the answer. Haste packages share the bucket, since they share the namespace. `metro-file-map` doesn't change. Its `Observations` is an interface, and Metro's record is a wider type that satisfies it, which is why that type wasn't exported. Two things keep this cheap: - Nothing is observed when no file can be given a Haste name - no `hasteImplModulePath` and `enableGlobalPackages: false`, which is Metro's default. The Haste map is empty then and always will be, so every lookup is a guaranteed miss, and recording one per bare specifier would be paying to observe something that can't change. `createModuleResolver` works this out from config, by the same rule `HastePlugin`'s filter uses. - The set is created on the first name recorded, and `haste` is `null` until then. Most resolutions are relative imports that never consult Haste. Allocating a set for each when Haste is enabled cost ~0.9MB on the benchmark app for ~0.2MB of entries. Nothing reads these yet. Invalidating from them needs `HastePlugin` to report which names changed binding in a batch - it's the only thing that knows, since a removed file's name is gone from the map by the time anyone else sees the change event. That's to follow, alongside invalidation generally. The empty module is resolved with `allowHaste: false`, so it observes no names and there's nothing to merge for it. ## Benchmark Resolution time in isolation, flag on in both columns, through the whole-resolution cache (see the test plan). No measurable cost either way - everything is within about half a percent, inside the run-to-run spread. | scenario | #1949 | this diff | #1949, Haste enabled | this diff, Haste enabled | |---|---|---|---|---| | first build (ms) | 112.4 | 111.8 | 111.2 | 110.6 | | rebuild, nothing changed (ms) | 7.75 | 7.78 | 7.81 | 7.79 | | after a source edit, the edited module's dependencies (µs) | 177 | 168 | 170 | 172 | | after a `package.json` change, re-resolving everything (ms) | 93.2 | 93.2 | 93.6 | 94.1 | | heap retained after resolving (MB) | 18.8 | 18.8 | | 19.1 | With Haste enabled, 27% of distinct resolutions look up a name (1.9k of 6.9k), one name each, over 72 distinct names. With it inert, no resolution carries a set. ## Test plan ``` yarn jest packages/metro-file-map packages/metro-resolver packages/metro/src/node-haste packages/metro/src/DeltaBundler packages/metro/src/integration_tests yarn flow check yarn typecheck-ts yarn verify-api-snapshots ``` New tests in `resolver-test.js`: nothing is observed when no file can be given a Haste name, a name is recorded when it's found and when it isn't, a deep import records only its package name, a relative import records nothing, and a global package name is recorded. The existing exact-set expectations gain `haste: null`. **Benchmark app** - a synthetic mid-size RN 0.87 app: 23 common dependencies (Reanimated, React Navigation, TanStack Query, lodash, etc.) and ~1,000 generated first-party modules, 60% of them in a workspace package outside `projectRoot` with its own `node_modules`. The iOS dev bundle is 2.8k modules, from a file map of 34k files. Timings replay the `resolveDependency` calls a real build of it makes against a fresh `DependencyGraph` per process, in interleaved rounds, with source and `package.json` edits made on disk and picked up by the watcher. "Haste enabled" is the same app with `enableGlobalPackages: true`, which gives its first-party packages Haste names. These timings isolate resolution - no transformation, serialisation or file reads are included. A real build of this app takes around 4s even with a warm transform cache, so resolution is about 2% of it. Changelog: Internal
1 parent 4437f0b commit f4682e7

4 files changed

Lines changed: 154 additions & 11 deletions

File tree

‎packages/metro/src/DeltaBundler/__tests__/resolver-test.js‎

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3028,6 +3028,7 @@ function dep(name: string): TransformResultDependency {
30283028
'package.json',
30293029
),
30303030
content: new Set(),
3031+
haste: null,
30313032
},
30323033
});
30333034
});
@@ -3074,6 +3075,7 @@ function dep(name: string): TransformResultDependency {
30743075
'node_modules/pkg/main.js',
30753076
),
30763077
content: new Set(),
3078+
haste: null,
30773079
},
30783080
});
30793081
});
@@ -3096,6 +3098,7 @@ function dep(name: string): TransformResultDependency {
30963098
),
30973099
// Retargeting the link changes what every path through it means
30983100
content: canonical('link'),
3101+
haste: null,
30993102
},
31003103
});
31013104
});
@@ -3121,6 +3124,7 @@ function dep(name: string): TransformResultDependency {
31213124
'package.json',
31223125
),
31233126
content: new Set(),
3127+
haste: null,
31243128
});
31253129
});
31263130

@@ -3160,6 +3164,7 @@ function dep(name: string): TransformResultDependency {
31603164
'package.json',
31613165
),
31623166
content: new Set(),
3167+
haste: null,
31633168
},
31643169
});
31653170
});
@@ -3172,6 +3177,110 @@ function dep(name: string): TransformResultDependency {
31723177
expect(second).toBe(first);
31733178
expect(second.unstable_observations?.existence.size).toBeGreaterThan(0);
31743179
});
3180+
describe('Haste names', () => {
3181+
const withHaste: InputConfigT = {
3182+
resolver: {
3183+
hasteImplModulePath: path.join(
3184+
__dirname,
3185+
'../__fixtures__/hasteImpl.js',
3186+
),
3187+
unstable_incrementalResolution: true,
3188+
},
3189+
};
3190+
3191+
test('are not observed when no file can be given one', async () => {
3192+
setMockFileSystem({
3193+
'index.js': '',
3194+
node_modules: {pkg: {'index.js': ''}},
3195+
});
3196+
resolver = await createResolver(incremental, 'ios');
3197+
// A bare specifier is looked up in the Haste map, which is empty and
3198+
// will stay that way
3199+
expect(
3200+
resolver.resolve(p('/root/index.js'), dep('pkg'))
3201+
.unstable_observations?.haste,
3202+
).toBe(null);
3203+
});
3204+
3205+
test('records a name that is found', async () => {
3206+
setMockFileSystem({
3207+
'index.js': '',
3208+
'hasteModule.js': '@providesModule hasteModule',
3209+
});
3210+
resolver = await createResolver(withHaste, 'ios');
3211+
const resolution = resolver.resolve(
3212+
p('/root/index.js'),
3213+
dep('hasteModule'),
3214+
);
3215+
expect(resolution.filePath).toBe(p('/root/hasteModule.js'));
3216+
expect(resolution.unstable_observations?.haste).toEqual(
3217+
new Set(['hasteModule']),
3218+
);
3219+
});
3220+
3221+
test('records a name that is not found, since it may appear later', async () => {
3222+
setMockFileSystem({
3223+
'index.js': '',
3224+
node_modules: {pkg: {'index.js': ''}},
3225+
});
3226+
resolver = await createResolver(withHaste, 'ios');
3227+
const resolution = resolver.resolve(p('/root/index.js'), dep('pkg'));
3228+
expect(resolution.filePath).toBe(
3229+
p('/root/node_modules/pkg/index.js'),
3230+
);
3231+
expect(resolution.unstable_observations?.haste).toEqual(
3232+
new Set(['pkg']),
3233+
);
3234+
});
3235+
3236+
test('records only the package name of a deep import', async () => {
3237+
setMockFileSystem({
3238+
'index.js': '',
3239+
node_modules: {pkg: {lib: {'thing.js': ''}}},
3240+
});
3241+
resolver = await createResolver(withHaste, 'ios');
3242+
expect(
3243+
resolver.resolve(p('/root/index.js'), dep('pkg/lib/thing'))
3244+
.unstable_observations?.haste,
3245+
).toEqual(new Set(['pkg']));
3246+
});
3247+
3248+
test('records nothing for a relative import, which does not consult Haste', async () => {
3249+
setMockFileSystem({'index.js': '', 'a.js': ''});
3250+
resolver = await createResolver(withHaste, 'ios');
3251+
expect(
3252+
resolver.resolve(p('/root/index.js'), dep('./a'))
3253+
.unstable_observations?.haste,
3254+
).toBe(null);
3255+
});
3256+
3257+
test('records a global package name', async () => {
3258+
setMockFileSystem({
3259+
'index.js': '',
3260+
aPackage: {
3261+
'package.json': JSON.stringify({name: 'aPackage'}),
3262+
'index.js': '',
3263+
},
3264+
});
3265+
resolver = await createResolver(
3266+
{
3267+
resolver: {
3268+
enableGlobalPackages: true,
3269+
unstable_incrementalResolution: true,
3270+
},
3271+
},
3272+
'ios',
3273+
);
3274+
const resolution = resolver.resolve(
3275+
p('/root/index.js'),
3276+
dep('aPackage'),
3277+
);
3278+
expect(resolution.filePath).toBe(p('/root/aPackage/index.js'));
3279+
expect(resolution.unstable_observations?.haste).toEqual(
3280+
new Set(['aPackage']),
3281+
);
3282+
});
3283+
});
31753284
});
31763285

31773286
describe('schemeResolvers', () => {

‎packages/metro/src/DeltaBundler/types.js‎

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -151,12 +151,19 @@ export type AllowOptionalDependenciesWithOptions = {
151151
export type AllowOptionalDependencies =
152152
boolean | AllowOptionalDependenciesWithOptions;
153153

154-
// What a resolution observed of the file system: canonical paths whose
155-
// addition or removal can change its result, and those whose modification
156-
// can too. Satisfies `metro-file-map`'s `Observations`, which populates it.
154+
// What a resolution observed. `existence` and `content` are canonical paths
155+
// whose addition or removal can change its result, and those whose
156+
// modification can too. They satisfy `metro-file-map`'s `Observations`, which
157+
// populates them.
158+
//
159+
// `haste` is every Haste module or package name the resolution looked up,
160+
// whether or not it was found, since one appearing later changes the result.
161+
// It is null if there were none, which is always the case when no file can be
162+
// given a Haste name - nothing is paid to observe what cannot change.
157163
export type ResolutionObservations = Readonly<{
158164
existence: Set<string>,
159165
content: Set<string>,
166+
haste: ?Set<string>,
160167
}>;
161168

162169
export type BundlerResolution = Readonly<{

‎packages/metro/src/node-haste/DependencyGraph/ModuleResolution.js‎

Lines changed: 30 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,9 @@ type Options = Readonly<{
4848
) => ReturnType<FileSystemLookup>,
4949
getHasteModulePath: (name: string, platform: ?string) => ?string,
5050
getHastePackagePath: (name: string, platform: ?string) => ?string,
51+
// Whether any file can be given a Haste name. If not, Haste lookups always
52+
// miss, and are not worth observing.
53+
isHasteEnabled: boolean,
5154
mainFields: ReadonlyArray<string>,
5255
getPackage: (packageJsonPath: string) => ?PackageJson,
5356
getPackageForModule: (
@@ -77,8 +80,8 @@ type Options = Readonly<{
7780

7881
// Every record is created here, with the same properties in the same order,
7982
// so that reading them stays monomorphic on the lookup hot path.
80-
function createObservations(): ResolutionObservations {
81-
return {existence: new Set(), content: new Set()};
83+
function createObservations(): {...ResolutionObservations} {
84+
return {existence: new Set(), content: new Set(), haste: null};
8285
}
8386

8487
export class ModuleResolver {
@@ -149,9 +152,21 @@ export class ModuleResolver {
149152
// The capabilities given to the resolver are bound to it, so that the
150153
// resolution context keeps its shape and a custom resolver records what
151154
// it looks up without having to know about it.
152-
const observations: ?ResolutionObservations = unstable_incrementalResolution
155+
const observations = unstable_incrementalResolution
153156
? createObservations()
154157
: null;
158+
// Most resolutions never consult Haste, so the set is created on demand.
159+
const observeHasteName =
160+
observations != null && this._options.isHasteEnabled
161+
? (name: string) => {
162+
let names = observations.haste;
163+
if (names == null) {
164+
names = new Set<string>();
165+
observations.haste = names;
166+
}
167+
names.add(name);
168+
}
169+
: null;
155170

156171
try {
157172
const result = Resolver.resolve(
@@ -187,10 +202,17 @@ export class ModuleResolver {
187202
? resolveAsset
188203
: (dirPath, assetName, extension) =>
189204
resolveAsset(dirPath, assetName, extension, observations),
190-
resolveHasteModule: (name: string) =>
191-
this._options.getHasteModulePath(name, platform),
192-
resolveHastePackage: (name: string) =>
193-
this._options.getHastePackagePath(name, platform),
205+
// A name is recorded whether or not it is found, and for any
206+
// platform, since `HastePlugin` falls back from the platform to
207+
// `native` and then to the generic module of that name.
208+
resolveHasteModule: (name: string) => {
209+
observeHasteName?.(name);
210+
return this._options.getHasteModulePath(name, platform);
211+
},
212+
resolveHastePackage: (name: string) => {
213+
observeHasteName?.(name);
214+
return this._options.getHastePackagePath(name, platform);
215+
},
194216
resolveRequest,
195217
schemeResolvers,
196218
sourceExts,
@@ -303,7 +325,7 @@ export class ModuleResolver {
303325
}
304326
// The empty module is resolved once and cached, so a resolution that
305327
// lands on it depends on what that resolution observed as well as on
306-
// what led here.
328+
// what led here. It is resolved without Haste, so observes no names.
307329
const emptyObservations = emptyModule.unstable_observations;
308330
if (emptyObservations != null) {
309331
for (const canonicalPath of emptyObservations.existence) {

‎packages/metro/src/node-haste/DependencyGraph/createModuleResolver.js‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,11 @@ export default function createModuleResolver({
6565
emptyModulePath: config.resolver.emptyModulePath,
6666
extraNodeModules: config.resolver.extraNodeModules,
6767
fileSystemLookup,
68+
// As `HastePlugin` decides which files to give a name: packages only if
69+
// global packages are enabled, and other files only with a `hasteImpl`.
70+
isHasteEnabled:
71+
config.resolver.enableGlobalPackages ||
72+
config.resolver.hasteImplModulePath != null,
6873
getHasteModulePath: (name, platform) =>
6974
hasteMap.getModule(name, platform, true),
7075
getHastePackagePath: (name, platform) =>

0 commit comments

Comments
 (0)