Skip to content

Commit fe0a1bc

Browse files
committed
Resolver: Observe Haste names under unstable_incrementalResolution
Stacked on #1949. ## Summary #1949 records what a resolution looks up in the file map. It doesn't record Haste lookups, which aren't paths - the resolver asks the Haste map for a module or package by name, for every bare specifier when `allowHaste` is set. A module of that name appearing later changes the result, so the name needs observing whether or not it was found. This adds `haste` to `ResolutionObservations`, the set of Haste names a resolution looked up. They're recorded in the two Haste closures that `ModuleResolver` already creates per resolution, so nothing else changes and the resolution context is untouched. It's the bare name that's recorded, not the platform - `HastePlugin` falls back from the platform to `native` and then to the generic module of that name, so a change to any of those bindings can change the answer. Haste packages share the bucket, since they share the namespace. `metro-file-map` doesn't change. Its `Observations` is an interface, and Metro's record is a wider type that satisfies it, which is why that type wasn't exported. Two things keep this cheap: - Nothing is observed when no file can be given a Haste name - no `hasteImplModulePath` and `enableGlobalPackages: false`, which is Metro's default. The Haste map is empty then and always will be, so every lookup is a guaranteed miss, and recording one per bare specifier would be paying to observe something that can't change. `createModuleResolver` works this out from config, by the same rule `HastePlugin`'s filter uses. - The set is created on the first name recorded, and `haste` is `null` until then. Most resolutions are relative imports that never consult Haste. Allocating a set for each when Haste is enabled cost ~0.9MB on the benchmark app for ~0.2MB of entries. Nothing reads these yet. Invalidating from them needs `HastePlugin` to report which names changed binding in a batch - it's the only thing that knows, since a removed file's name is gone from the map by the time anyone else sees the change event. That's to follow, alongside invalidation generally. The empty module is resolved with `allowHaste: false`, so it observes no names and there's nothing to merge for it. ## Benchmark Resolution time in isolation, flag on in both columns, through the whole-resolution cache (see the test plan). No measurable cost either way - everything is within about half a percent, inside the run-to-run spread. | scenario | #1949 | this diff | #1949, Haste enabled | this diff, Haste enabled | |---|---|---|---|---| | first build (ms) | 112.4 | 111.8 | 111.2 | 110.6 | | rebuild, nothing changed (ms) | 7.75 | 7.78 | 7.81 | 7.79 | | after a source edit, the edited module's dependencies (µs) | 177 | 168 | 170 | 172 | | after a `package.json` change, re-resolving everything (ms) | 93.2 | 93.2 | 93.6 | 94.1 | | heap retained after resolving (MB) | 18.8 | 18.8 | | 19.1 | With Haste enabled, 27% of distinct resolutions look up a name (1.9k of 6.9k), one name each, over 72 distinct names. With it inert, no resolution carries a set. ## Test plan ``` yarn jest packages/metro-file-map packages/metro-resolver packages/metro/src/node-haste packages/metro/src/DeltaBundler packages/metro/src/integration_tests yarn flow check yarn typecheck-ts yarn verify-api-snapshots ``` New tests in `resolver-test.js`: nothing is observed when no file can be given a Haste name, a name is recorded when it's found and when it isn't, a deep import records only its package name, a relative import records nothing, and a global package name is recorded. The existing exact-set expectations gain `haste: null`. **Benchmark app** - a synthetic mid-size RN 0.87 app: 23 common dependencies (Reanimated, React Navigation, TanStack Query, lodash, etc.) and ~1,000 generated first-party modules, 60% of them in a workspace package outside `projectRoot` with its own `node_modules`. The iOS dev bundle is 2.8k modules, from a file map of 34k files. Timings replay the `resolveDependency` calls a real build of it makes against a fresh `DependencyGraph` per process, in interleaved rounds, with source and `package.json` edits made on disk and picked up by the watcher. "Haste enabled" is the same app with `enableGlobalPackages: true`, which gives its first-party packages Haste names. These timings isolate resolution - no transformation, serialisation or file reads are included. A real build of this app takes around 4s even with a warm transform cache, so resolution is about 2% of it. Changelog: Internal
1 parent 4a700b3 commit fe0a1bc

4 files changed

Lines changed: 148 additions & 8 deletions

File tree

‎packages/metro/src/DeltaBundler/__tests__/resolver-test.js‎

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3040,6 +3040,7 @@ function dep(name: string): TransformResultDependency {
30403040
'package.json',
30413041
),
30423042
content: new Set(),
3043+
haste: null,
30433044
},
30443045
});
30453046
});
@@ -3085,6 +3086,7 @@ function dep(name: string): TransformResultDependency {
30853086
'node_modules/pkg/main.js',
30863087
),
30873088
content: new Set(),
3089+
haste: null,
30883090
},
30893091
});
30903092
});
@@ -3107,6 +3109,7 @@ function dep(name: string): TransformResultDependency {
31073109
),
31083110
// Retargeting the link changes what every path through it means
31093111
content: canonical('link'),
3112+
haste: null,
31103113
},
31113114
});
31123115
});
@@ -3132,6 +3135,7 @@ function dep(name: string): TransformResultDependency {
31323135
'package.json',
31333136
),
31343137
content: new Set(),
3138+
haste: null,
31353139
});
31363140
});
31373141

@@ -3171,6 +3175,7 @@ function dep(name: string): TransformResultDependency {
31713175
'package.json',
31723176
),
31733177
content: new Set(),
3178+
haste: null,
31743179
},
31753180
});
31763181
});
@@ -3183,6 +3188,110 @@ function dep(name: string): TransformResultDependency {
31833188
expect(second).toBe(first);
31843189
expect(second.unstable_observations?.existence.size).toBeGreaterThan(0);
31853190
});
3191+
describe('Haste names', () => {
3192+
const withHaste: InputConfigT = {
3193+
resolver: {
3194+
hasteImplModulePath: path.join(
3195+
__dirname,
3196+
'../__fixtures__/hasteImpl.js',
3197+
),
3198+
unstable_incrementalResolution: true,
3199+
},
3200+
};
3201+
3202+
test('are not observed when no file can be given one', async () => {
3203+
setMockFileSystem({
3204+
'index.js': '',
3205+
node_modules: {pkg: {'index.js': ''}},
3206+
});
3207+
resolver = await createResolver(incremental, 'ios');
3208+
// A bare specifier is looked up in the Haste map, which is empty and
3209+
// will stay that way
3210+
expect(
3211+
resolver.resolve(p('/root/index.js'), dep('pkg'))
3212+
.unstable_observations?.haste,
3213+
).toBe(null);
3214+
});
3215+
3216+
test('records a name that is found', async () => {
3217+
setMockFileSystem({
3218+
'index.js': '',
3219+
'hasteModule.js': '@providesModule hasteModule',
3220+
});
3221+
resolver = await createResolver(withHaste, 'ios');
3222+
const resolution = resolver.resolve(
3223+
p('/root/index.js'),
3224+
dep('hasteModule'),
3225+
);
3226+
expect(resolution.filePath).toBe(p('/root/hasteModule.js'));
3227+
expect(resolution.unstable_observations?.haste).toEqual(
3228+
new Set(['hasteModule']),
3229+
);
3230+
});
3231+
3232+
test('records a name that is not found, since it may appear later', async () => {
3233+
setMockFileSystem({
3234+
'index.js': '',
3235+
node_modules: {pkg: {'index.js': ''}},
3236+
});
3237+
resolver = await createResolver(withHaste, 'ios');
3238+
const resolution = resolver.resolve(p('/root/index.js'), dep('pkg'));
3239+
expect(resolution.filePath).toBe(
3240+
p('/root/node_modules/pkg/index.js'),
3241+
);
3242+
expect(resolution.unstable_observations?.haste).toEqual(
3243+
new Set(['pkg']),
3244+
);
3245+
});
3246+
3247+
test('records only the package name of a deep import', async () => {
3248+
setMockFileSystem({
3249+
'index.js': '',
3250+
node_modules: {pkg: {lib: {'thing.js': ''}}},
3251+
});
3252+
resolver = await createResolver(withHaste, 'ios');
3253+
expect(
3254+
resolver.resolve(p('/root/index.js'), dep('pkg/lib/thing'))
3255+
.unstable_observations?.haste,
3256+
).toEqual(new Set(['pkg']));
3257+
});
3258+
3259+
test('records nothing for a relative import, which does not consult Haste', async () => {
3260+
setMockFileSystem({'index.js': '', 'a.js': ''});
3261+
resolver = await createResolver(withHaste, 'ios');
3262+
expect(
3263+
resolver.resolve(p('/root/index.js'), dep('./a'))
3264+
.unstable_observations?.haste,
3265+
).toBe(null);
3266+
});
3267+
3268+
test('records a global package name', async () => {
3269+
setMockFileSystem({
3270+
'index.js': '',
3271+
aPackage: {
3272+
'package.json': JSON.stringify({name: 'aPackage'}),
3273+
'index.js': '',
3274+
},
3275+
});
3276+
resolver = await createResolver(
3277+
{
3278+
resolver: {
3279+
enableGlobalPackages: true,
3280+
unstable_incrementalResolution: true,
3281+
},
3282+
},
3283+
'ios',
3284+
);
3285+
const resolution = resolver.resolve(
3286+
p('/root/index.js'),
3287+
dep('aPackage'),
3288+
);
3289+
expect(resolution.filePath).toBe(p('/root/aPackage/index.js'));
3290+
expect(resolution.unstable_observations?.haste).toEqual(
3291+
new Set(['aPackage']),
3292+
);
3293+
});
3294+
});
31863295
});
31873296

31883297
describe('schemeResolvers', () => {

‎packages/metro/src/DeltaBundler/types.js‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -163,10 +163,13 @@ export type AllowOptionalDependencies =
163163
boolean | AllowOptionalDependenciesWithOptions;
164164

165165
// Canonical paths whose addition or removal (`existence`) or modification
166-
// (`content`) can change a resolution's result.
166+
// (`content`) can change a resolution's result, and every Haste name it looked
167+
// up, found or not (`haste`). `haste` is null if there were none, which is
168+
// always so when no file can have a Haste name.
167169
export type ResolutionObservations = Readonly<{
168170
existence: Set<string>,
169171
content: Set<string>,
172+
haste: ?Set<string>,
170173
}>;
171174

172175
export type BundlerResolution = Readonly<{

‎packages/metro/src/node-haste/DependencyGraph/ModuleResolution.js‎

Lines changed: 30 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,9 @@ type Options = Readonly<{
4848
) => ReturnType<FileSystemLookup>,
4949
getHasteModulePath: (name: string, platform: ?string) => ?string,
5050
getHastePackagePath: (name: string, platform: ?string) => ?string,
51+
// Whether any file can be given a Haste name. If not, Haste lookups always
52+
// miss, and are not worth observing.
53+
isHasteEnabled: boolean,
5154
mainFields: ReadonlyArray<string>,
5255
getPackage: (packageJsonPath: string) => ?PackageJson,
5356
getPackageForModule: (
@@ -76,8 +79,8 @@ type Options = Readonly<{
7679
}>;
7780

7881
// Same shape for every record, to keep lookups monomorphic.
79-
function createObservations(): ResolutionObservations {
80-
return {existence: new Set(), content: new Set()};
82+
function createObservations(): {...ResolutionObservations} {
83+
return {existence: new Set(), content: new Set(), haste: null};
8184
}
8285

8386
export class ModuleResolver {
@@ -210,7 +213,7 @@ export class ModuleResolver {
210213
allowHaste: boolean,
211214
platform: string | null,
212215
resolverOptions: ResolverInputOptions,
213-
observations: ?ResolutionObservations,
216+
observations: ?{...ResolutionObservations},
214217
): Resolution {
215218
const {
216219
assetExts,
@@ -232,6 +235,18 @@ export class ModuleResolver {
232235
unstable_enablePackageExports,
233236
unstable_incrementalResolution,
234237
} = this._options;
238+
// Most resolutions never consult Haste, so the set is created on demand.
239+
const observeHasteName =
240+
observations != null && this._options.isHasteEnabled
241+
? (name: string) => {
242+
let names = observations.haste;
243+
if (names == null) {
244+
names = new Set<string>();
245+
observations.haste = names;
246+
}
247+
names.add(name);
248+
}
249+
: null;
235250

236251
return Resolver.resolve(
237252
createDefaultContext(
@@ -265,10 +280,17 @@ export class ModuleResolver {
265280
? resolveAsset
266281
: (dirPath, assetName, extension) =>
267282
resolveAsset(dirPath, assetName, extension, observations),
268-
resolveHasteModule: (name: string) =>
269-
this._options.getHasteModulePath(name, platform),
270-
resolveHastePackage: (name: string) =>
271-
this._options.getHastePackagePath(name, platform),
283+
// A name is recorded whether or not it is found, and for any
284+
// platform, since `HastePlugin` falls back from the platform to
285+
// `native` and then to the generic module of that name.
286+
resolveHasteModule: (name: string) => {
287+
observeHasteName?.(name);
288+
return this._options.getHasteModulePath(name, platform);
289+
},
290+
resolveHastePackage: (name: string) => {
291+
observeHasteName?.(name);
292+
return this._options.getHastePackagePath(name, platform);
293+
},
272294
resolveRequest,
273295
schemeResolvers,
274296
sourceExts,
@@ -319,6 +341,7 @@ export class ModuleResolver {
319341
return emptyModule;
320342
}
321343
// The empty module's resolution is cached, so merge in what it observed.
344+
// It is resolved without Haste, so has no names to merge.
322345
const emptyObservations = emptyModule.unstable_observations;
323346
if (emptyObservations != null) {
324347
for (const canonicalPath of emptyObservations.existence) {

‎packages/metro/src/node-haste/DependencyGraph/createModuleResolver.js‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,11 @@ export default function createModuleResolver({
6464
emptyModulePath: config.resolver.emptyModulePath,
6565
extraNodeModules: config.resolver.extraNodeModules,
6666
fileSystemLookup,
67+
// As `HastePlugin` decides which files to give a name: packages only if
68+
// global packages are enabled, and other files only with a `hasteImpl`.
69+
isHasteEnabled:
70+
config.resolver.enableGlobalPackages ||
71+
config.resolver.hasteImplModulePath != null,
6772
getHasteModulePath: (name, platform) =>
6873
hasteMap.getModule(name, platform, true),
6974
getHastePackagePath: (name, platform) =>

0 commit comments

Comments
 (0)