Skip to content

Commit 467b106

Browse files
committed
Resolver: Observe Haste names under unstable_incrementalResolution
Stacked on #1949. ## Summary #1949 records what a resolution looks up in the file map. It doesn't record Haste lookups, which aren't paths - the resolver asks the Haste map for a module or package by name, for every bare specifier when `allowHaste` is set. A module of that name appearing later changes the result, so the name needs observing whether or not it was found. This adds `haste` to `ResolutionObservations`, the set of Haste names a resolution looked up. They're recorded in the two Haste closures that `ModuleResolver` already creates per resolution, so nothing else changes and the resolution context is untouched. It's the bare name that's recorded, not the platform - `HastePlugin` falls back from the platform to `native` and then to the generic module of that name, so a change to any of those bindings can change the answer. Haste packages share the bucket, since they share the namespace. `metro-file-map` doesn't change. Its `Observations` is an interface, and Metro's record is a wider type that satisfies it, which is why that type wasn't exported. Two things keep this cheap: - Nothing is observed when no file can be given a Haste name - no `hasteImplModulePath` and `enableGlobalPackages: false`, which is Metro's default. The Haste map is empty then and always will be, so every lookup is a guaranteed miss, and recording one per bare specifier would be paying to observe something that can't change. `createModuleResolver` works this out from config, by the same rule `HastePlugin`'s filter uses. - The set is created on the first name recorded, and `haste` is `null` until then. Most resolutions are relative imports that never consult Haste. Allocating a set for each when Haste is enabled cost ~0.9MB on the benchmark app for ~0.2MB of entries. Nothing reads these yet. Invalidating from them needs `HastePlugin` to report which names changed binding in a batch - it's the only thing that knows, since a removed file's name is gone from the map by the time anyone else sees the change event. That's to follow, alongside invalidation generally. The empty module is resolved with `allowHaste: false`, so it observes no names and there's nothing to merge for it. ## Benchmark Resolution time in isolation, flag on in both columns, through the whole-resolution cache (see the test plan). No measurable cost either way - everything is within about half a percent, inside the run-to-run spread. | scenario | #1949 | this diff | #1949, Haste enabled | this diff, Haste enabled | |---|---|---|---|---| | first build (ms) | 112.4 | 111.8 | 111.2 | 110.6 | | rebuild, nothing changed (ms) | 7.75 | 7.78 | 7.81 | 7.79 | | after a source edit, the edited module's dependencies (µs) | 177 | 168 | 170 | 172 | | after a `package.json` change, re-resolving everything (ms) | 93.2 | 93.2 | 93.6 | 94.1 | | heap retained after resolving (MB) | 18.8 | 18.8 | | 19.1 | With Haste enabled, 27% of distinct resolutions look up a name (1.9k of 6.9k), one name each, over 72 distinct names. With it inert, no resolution carries a set. ## Test plan ``` yarn jest packages/metro-file-map packages/metro-resolver packages/metro/src/node-haste packages/metro/src/DeltaBundler packages/metro/src/integration_tests yarn flow check yarn typecheck-ts yarn verify-api-snapshots ``` New tests in `resolver-test.js`: nothing is observed when no file can be given a Haste name, a name is recorded when it's found and when it isn't, a deep import records only its package name, a relative import records nothing, and a global package name is recorded. The existing exact-set expectations gain `haste: null`. **Benchmark app** - a synthetic mid-size RN 0.87 app: 23 common dependencies (Reanimated, React Navigation, TanStack Query, lodash, etc.) and ~1,000 generated first-party modules, 60% of them in a workspace package outside `projectRoot` with its own `node_modules`. The iOS dev bundle is 2.8k modules, from a file map of 34k files. Timings replay the `resolveDependency` calls a real build of it makes against a fresh `DependencyGraph` per process, in interleaved rounds, with source and `package.json` edits made on disk and picked up by the watcher. "Haste enabled" is the same app with `enableGlobalPackages: true`, which gives its first-party packages Haste names. These timings isolate resolution - no transformation, serialisation or file reads are included. A real build of this app takes around 4s even with a warm transform cache, so resolution is about 2% of it. Changelog: Internal
1 parent 4e24402 commit 467b106

4 files changed

Lines changed: 154 additions & 11 deletions

File tree

‎packages/metro/src/DeltaBundler/__tests__/resolver-test.js‎

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2886,6 +2886,7 @@ function dep(name: string): TransformResultDependency {
28862886
'package.json',
28872887
),
28882888
content: new Set(),
2889+
haste: null,
28892890
},
28902891
});
28912892
});
@@ -2930,6 +2931,7 @@ function dep(name: string): TransformResultDependency {
29302931
'node_modules/pkg/main.js',
29312932
),
29322933
content: new Set(),
2934+
haste: null,
29332935
},
29342936
});
29352937
});
@@ -2952,6 +2954,7 @@ function dep(name: string): TransformResultDependency {
29522954
),
29532955
// Retargeting the link changes what every path through it means
29542956
content: canonical('link'),
2957+
haste: null,
29552958
},
29562959
});
29572960
});
@@ -2977,6 +2980,7 @@ function dep(name: string): TransformResultDependency {
29772980
'package.json',
29782981
),
29792982
content: new Set(),
2983+
haste: null,
29802984
});
29812985
});
29822986

@@ -3016,6 +3020,7 @@ function dep(name: string): TransformResultDependency {
30163020
'package.json',
30173021
),
30183022
content: new Set(),
3023+
haste: null,
30193024
},
30203025
});
30213026
});
@@ -3028,6 +3033,110 @@ function dep(name: string): TransformResultDependency {
30283033
expect(second).toBe(first);
30293034
expect(second.unstable_observations?.existence.size).toBeGreaterThan(0);
30303035
});
3036+
describe('Haste names', () => {
3037+
const withHaste: InputConfigT = {
3038+
resolver: {
3039+
hasteImplModulePath: path.join(
3040+
__dirname,
3041+
'../__fixtures__/hasteImpl.js',
3042+
),
3043+
unstable_incrementalResolution: true,
3044+
},
3045+
};
3046+
3047+
test('are not observed when no file can be given one', async () => {
3048+
setMockFileSystem({
3049+
'index.js': '',
3050+
node_modules: {pkg: {'index.js': ''}},
3051+
});
3052+
resolver = await createResolver(incremental, 'ios');
3053+
// A bare specifier is looked up in the Haste map, which is empty and
3054+
// will stay that way
3055+
expect(
3056+
resolver.resolve(p('/root/index.js'), dep('pkg'))
3057+
.unstable_observations?.haste,
3058+
).toBe(null);
3059+
});
3060+
3061+
test('records a name that is found', async () => {
3062+
setMockFileSystem({
3063+
'index.js': '',
3064+
'hasteModule.js': '@providesModule hasteModule',
3065+
});
3066+
resolver = await createResolver(withHaste, 'ios');
3067+
const resolution = resolver.resolve(
3068+
p('/root/index.js'),
3069+
dep('hasteModule'),
3070+
);
3071+
expect(resolution.filePath).toBe(p('/root/hasteModule.js'));
3072+
expect(resolution.unstable_observations?.haste).toEqual(
3073+
new Set(['hasteModule']),
3074+
);
3075+
});
3076+
3077+
test('records a name that is not found, since it may appear later', async () => {
3078+
setMockFileSystem({
3079+
'index.js': '',
3080+
node_modules: {pkg: {'index.js': ''}},
3081+
});
3082+
resolver = await createResolver(withHaste, 'ios');
3083+
const resolution = resolver.resolve(p('/root/index.js'), dep('pkg'));
3084+
expect(resolution.filePath).toBe(
3085+
p('/root/node_modules/pkg/index.js'),
3086+
);
3087+
expect(resolution.unstable_observations?.haste).toEqual(
3088+
new Set(['pkg']),
3089+
);
3090+
});
3091+
3092+
test('records only the package name of a deep import', async () => {
3093+
setMockFileSystem({
3094+
'index.js': '',
3095+
node_modules: {pkg: {lib: {'thing.js': ''}}},
3096+
});
3097+
resolver = await createResolver(withHaste, 'ios');
3098+
expect(
3099+
resolver.resolve(p('/root/index.js'), dep('pkg/lib/thing'))
3100+
.unstable_observations?.haste,
3101+
).toEqual(new Set(['pkg']));
3102+
});
3103+
3104+
test('records nothing for a relative import, which does not consult Haste', async () => {
3105+
setMockFileSystem({'index.js': '', 'a.js': ''});
3106+
resolver = await createResolver(withHaste, 'ios');
3107+
expect(
3108+
resolver.resolve(p('/root/index.js'), dep('./a'))
3109+
.unstable_observations?.haste,
3110+
).toBe(null);
3111+
});
3112+
3113+
test('records a global package name', async () => {
3114+
setMockFileSystem({
3115+
'index.js': '',
3116+
aPackage: {
3117+
'package.json': JSON.stringify({name: 'aPackage'}),
3118+
'index.js': '',
3119+
},
3120+
});
3121+
resolver = await createResolver(
3122+
{
3123+
resolver: {
3124+
enableGlobalPackages: true,
3125+
unstable_incrementalResolution: true,
3126+
},
3127+
},
3128+
'ios',
3129+
);
3130+
const resolution = resolver.resolve(
3131+
p('/root/index.js'),
3132+
dep('aPackage'),
3133+
);
3134+
expect(resolution.filePath).toBe(p('/root/aPackage/index.js'));
3135+
expect(resolution.unstable_observations?.haste).toEqual(
3136+
new Set(['aPackage']),
3137+
);
3138+
});
3139+
});
30313140
});
30323141

30333142
describe('schemeResolvers', () => {

‎packages/metro/src/DeltaBundler/types.js‎

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -151,12 +151,19 @@ export type AllowOptionalDependenciesWithOptions = {
151151
export type AllowOptionalDependencies =
152152
boolean | AllowOptionalDependenciesWithOptions;
153153

154-
// What a resolution observed of the file system: canonical paths whose
155-
// addition or removal can change its result, and those whose modification
156-
// can too. Satisfies `metro-file-map`'s `Observations`, which populates it.
154+
// What a resolution observed. `existence` and `content` are canonical paths
155+
// whose addition or removal can change its result, and those whose
156+
// modification can too. They satisfy `metro-file-map`'s `Observations`, which
157+
// populates them.
158+
//
159+
// `haste` is every Haste module or package name the resolution looked up,
160+
// whether or not it was found, since one appearing later changes the result.
161+
// It is null if there were none, which is always the case when no file can be
162+
// given a Haste name - nothing is paid to observe what cannot change.
157163
export type ResolutionObservations = Readonly<{
158164
existence: Set<string>,
159165
content: Set<string>,
166+
haste: ?Set<string>,
160167
}>;
161168

162169
export type BundlerResolution = Readonly<{

‎packages/metro/src/node-haste/DependencyGraph/ModuleResolution.js‎

Lines changed: 30 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,9 @@ type Options = Readonly<{
4848
) => ReturnType<FileSystemLookup>,
4949
getHasteModulePath: (name: string, platform: ?string) => ?string,
5050
getHastePackagePath: (name: string, platform: ?string) => ?string,
51+
// Whether any file can be given a Haste name. If not, Haste lookups always
52+
// miss, and are not worth observing.
53+
isHasteEnabled: boolean,
5154
mainFields: ReadonlyArray<string>,
5255
getPackage: (packageJsonPath: string) => ?PackageJson,
5356
getPackageForModule: (
@@ -77,8 +80,8 @@ type Options = Readonly<{
7780

7881
// Every record is created here, with the same properties in the same order,
7982
// so that reading them stays monomorphic on the lookup hot path.
80-
function createObservations(): ResolutionObservations {
81-
return {existence: new Set(), content: new Set()};
83+
function createObservations(): {...ResolutionObservations} {
84+
return {existence: new Set(), content: new Set(), haste: null};
8285
}
8386

8487
export class ModuleResolver {
@@ -149,9 +152,21 @@ export class ModuleResolver {
149152
// The capabilities given to the resolver are bound to it, so that the
150153
// resolution context keeps its shape and a custom resolver records what
151154
// it looks up without having to know about it.
152-
const observations: ?ResolutionObservations = unstable_incrementalResolution
155+
const observations = unstable_incrementalResolution
153156
? createObservations()
154157
: null;
158+
// Most resolutions never consult Haste, so the set is created on demand.
159+
const observeHasteName =
160+
observations != null && this._options.isHasteEnabled
161+
? (name: string) => {
162+
let names = observations.haste;
163+
if (names == null) {
164+
names = new Set<string>();
165+
observations.haste = names;
166+
}
167+
names.add(name);
168+
}
169+
: null;
155170

156171
try {
157172
const result = Resolver.resolve(
@@ -187,10 +202,17 @@ export class ModuleResolver {
187202
? resolveAsset
188203
: (dirPath, assetName, extension) =>
189204
resolveAsset(dirPath, assetName, extension, observations),
190-
resolveHasteModule: (name: string) =>
191-
this._options.getHasteModulePath(name, platform),
192-
resolveHastePackage: (name: string) =>
193-
this._options.getHastePackagePath(name, platform),
205+
// A name is recorded whether or not it is found, and for any
206+
// platform, since `HastePlugin` falls back from the platform to
207+
// `native` and then to the generic module of that name.
208+
resolveHasteModule: (name: string) => {
209+
observeHasteName?.(name);
210+
return this._options.getHasteModulePath(name, platform);
211+
},
212+
resolveHastePackage: (name: string) => {
213+
observeHasteName?.(name);
214+
return this._options.getHastePackagePath(name, platform);
215+
},
194216
resolveRequest,
195217
schemeResolvers,
196218
sourceExts,
@@ -303,7 +325,7 @@ export class ModuleResolver {
303325
}
304326
// The empty module is resolved once and cached, so a resolution that
305327
// lands on it depends on what that resolution observed as well as on
306-
// what led here.
328+
// what led here. It is resolved without Haste, so observes no names.
307329
const emptyObservations = emptyModule.unstable_observations;
308330
if (emptyObservations != null) {
309331
for (const canonicalPath of emptyObservations.existence) {

‎packages/metro/src/node-haste/DependencyGraph/createModuleResolver.js‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,11 @@ export default function createModuleResolver({
6565
emptyModulePath: config.resolver.emptyModulePath,
6666
extraNodeModules: config.resolver.extraNodeModules,
6767
fileSystemLookup,
68+
// As `HastePlugin` decides which files to give a name: packages only if
69+
// global packages are enabled, and other files only with a `hasteImpl`.
70+
isHasteEnabled:
71+
config.resolver.enableGlobalPackages ||
72+
config.resolver.hasteImplModulePath != null,
6873
getHasteModulePath: (name, platform) =>
6974
hasteMap.getModule(name, platform, true),
7075
getHastePackagePath: (name, platform) =>

0 commit comments

Comments
 (0)