Skip to content

Commit d56f1cf

Browse files
committed
Resolver: Resolve the package of a directory once for all extension candidates
Stacked on #1944. ## Summary `resolveFile` tries each candidate extension in turn, and for each one `resolveSourceFileForExt` calls `redirectModulePath`, which asks `context.getPackageForModule` for the package scope of the candidate path. Every candidate in a directory belongs to the same package, so that's the same question asked once per extension - and because Metro's `PackageCache` memoises by module path, each distinct candidate string misses the first time it's seen, mostly for files that don't exist. This asks once, up front, for the package of the files in the directory, and shares it with every candidate via the source file context. `redirectPackageSubpath` is the part of `redirectModulePath` that applies `mainFields` redirections when the containing package is already known. `redirectModulePath` delegates to it, and behaves as before for its other callers. The package of the files in a directory isn't always the package of the directory itself - a `node_modules` directory belongs to whatever contains it while the files in it belong to none, and a symlink can change which directory along the way is named `node_modules`. So `getPackageScopeOfFiles` asks about a stand-in file in the directory, `.metro-package-scope`, which gets the same answer any real candidate there would. If that name happens to exist as a package root, the answer is about that package instead, which shows in its package-relative path, so it tries another name, and throws if `getPackageForModule` keeps answering about something else. `package-scope-test.js` pins the sequence of `getPackageForModule` calls the resolver makes for the common specifier shapes, so that the call count is a tested property rather than incidental. Two `resolver-test.js` cases pin the package scope of files reached through a symlink to or from a `node_modules` directory, where the directory and the files in it disagree. This also removes the `@deprecated` tag from `getPackageForModule` - the resolver depends on it, and its doc now says what it returns for symlinked and missing paths. ## Benchmark This cuts package scope lookups 3.7x, and takes 28% off first-pass resolution time, measured in isolation on our benchmark app (described in the test plan). 7 interleaved rounds. Resolver output is byte-identical. | metric | #1944 | this diff | |---|---|---| | `getPackageForModule` calls per pass | 64k | **17k (3.7x fewer)** | | first pass after startup (ms) | 119.6 | **85.7 (−28.3%)** | | warm pass (ms) | 55.2 | **53.5 (−3.1%)** | | after a source change (ms) | 63.0 | **60.9 (−3.4%)** | | after a `package.json` change (ms) | 68.4 | **64.0 (−6.4%)** | The first pass is where this lands, because that's where every distinct candidate path is a `PackageCache` miss and a walk up the tree. ## Test plan ``` yarn jest packages/metro-resolver packages/metro-file-map packages/metro/src/node-haste packages/metro/src/DeltaBundler/__tests__/resolver-test.js yarn flow check yarn verify-api-snapshots ``` **Benchmark app** - a synthetic mid-size RN 0.87 app: 23 common dependencies (Reanimated, React Navigation, TanStack Query, lodash, etc.) and ~1,000 generated first-party modules, 60% of them in a workspace package outside `projectRoot` with its own `node_modules`. The iOS dev bundle is 2.8k modules, from a file map of 34k files. Timings replay the resolutions a real build of it makes against a fresh `DependencyGraph` per process, in interleaved rounds. These timings isolate resolution - no transformation, serialisation or file reads are included. A real build of this app takes around 4s even with a warm transform cache, so resolution is about 2% of it. Changelog: Internal
1 parent 215031a commit d56f1cf

5 files changed

Lines changed: 412 additions & 28 deletions

File tree

‎packages/metro-resolver/src/PackageResolve.js‎

Lines changed: 51 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -106,8 +106,6 @@ export function redirectModulePath(
106106
return modulePath;
107107
}
108108

109-
let redirectedPath;
110-
111109
if (modulePath.startsWith('.') || isModulePathAbsolute) {
112110
const packageRelativeModulePath = isModulePathAbsolute
113111
? // If the module path is absolute, containingPackage is relative to it
@@ -120,40 +118,68 @@ export function redirectModulePath(
120118
path.dirname(containingPackage.packageRelativePath),
121119
modulePath,
122120
);
123-
124-
redirectedPath = matchSubpathFromMainFields(
125-
// Use prefixed POSIX path for lookup in package.json
126-
'./' + systemToPosixPath(packageRelativeModulePath),
127-
containingPackage.packageJson,
128-
mainFields,
129-
);
130-
131-
if (typeof redirectedPath === 'string') {
132-
// BRITTLE ASSUMPTION: This is always treated as a package-relative path
133-
// and is converted back, even if the redirected path is a specifier
134-
// referring to another package.
135-
redirectedPath = path.isAbsolute(redirectedPath)
136-
? path.normalize(redirectedPath)
137-
: path.join(containingPackage.rootPath, redirectedPath);
138-
}
139-
} else {
140-
// Otherwise, `modulePath` may be an unprefixed relative path or a bare
141-
// specifier (can also be an absolute specifier prefixed with a URL scheme).
142-
// This is used only by the "browser" spec.
143-
redirectedPath = matchSubpathFromMainFields(
121+
return redirectPackageSubpath(
122+
context,
144123
modulePath,
145-
containingPackage.packageJson,
146-
mainFields,
124+
containingPackage,
125+
packageRelativeModulePath,
147126
);
148127
}
149128

129+
// Otherwise, `modulePath` may be an unprefixed relative path or a bare
130+
// specifier (can also be an absolute specifier prefixed with a URL scheme).
131+
// This is used only by the "browser" spec.
132+
const redirectedPath = matchSubpathFromMainFields(
133+
modulePath,
134+
containingPackage.packageJson,
135+
mainFields,
136+
);
137+
150138
if (redirectedPath != null) {
151139
return redirectedPath;
152140
}
153141

154142
return modulePath;
155143
}
156144

145+
/**
146+
* `redirectModulePath` for a module path whose containing package is already
147+
* known. `packageRelativePath` is the path of the module relative to the
148+
* package root, with system separators. Returns `false` if the module should
149+
* be ignored, and `modulePath` if no `package.json` mapping is matched.
150+
*/
151+
export function redirectPackageSubpath(
152+
context: Readonly<{
153+
mainFields: ResolutionContext['mainFields'],
154+
...
155+
}>,
156+
modulePath: string,
157+
containingPackage: Readonly<{
158+
packageJson: PackageInfo['packageJson'],
159+
rootPath: PackageInfo['rootPath'],
160+
...
161+
}>,
162+
packageRelativePath: string,
163+
): string | false {
164+
const redirectedPath = matchSubpathFromMainFields(
165+
// Use prefixed POSIX path for lookup in package.json
166+
'./' + systemToPosixPath(packageRelativePath),
167+
containingPackage.packageJson,
168+
context.mainFields,
169+
);
170+
171+
if (typeof redirectedPath === 'string') {
172+
// BRITTLE ASSUMPTION: This is always treated as a package-relative path
173+
// and is converted back, even if the redirected path is a specifier
174+
// referring to another package.
175+
return path.isAbsolute(redirectedPath)
176+
? path.normalize(redirectedPath)
177+
: path.join(containingPackage.rootPath, redirectedPath);
178+
}
179+
180+
return redirectedPath === false ? false : modulePath;
181+
}
182+
157183
/**
158184
* Get the mapped replacement for the given subpath defined by matching
159185
* `mainFields` entries in the passed `package.json`
Lines changed: 226 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,226 @@
1+
/**
2+
* Copyright (c) Meta Platforms, Inc. and affiliates.
3+
*
4+
* This source code is licensed under the MIT license found in the
5+
* LICENSE file in the root directory of this source tree.
6+
*
7+
* @flow strict-local
8+
* @format
9+
* @oncall react_native
10+
*/
11+
12+
import type {PackageForModule, ResolutionContext} from '../types';
13+
14+
import * as Resolver from '../index';
15+
import {
16+
createPackageAccessors,
17+
createResolutionContext,
18+
posixToSystemPath as p,
19+
} from './utils';
20+
21+
// The resolver looks up package scopes through `getPackageForModule`. These
22+
// tests pin down which paths it asks about for each kind of specifier, so
23+
// that changes to the number of lookups per resolution are deliberate. Metro
24+
// memoizes the answer per path, so a repeated path is cheap and a new one is
25+
// not: in particular, the files in a directory are looked up once, through a
26+
// stand-in file name, for all of their extension candidates.
27+
describe('package scope lookups', () => {
28+
// The path the resolver asks about for the files in a directory
29+
const filesIn = (dirPath: string) => p(dirPath + '/.metro-package-scope');
30+
31+
const fileMap = {
32+
[p('/root/package.json')]: JSON.stringify({name: 'root'}),
33+
[p('/root/src/main.js')]: '',
34+
[p('/root/src/foo.js')]: '',
35+
[p('/root/src/lib/index.js')]: '',
36+
[p('/root/node_modules/pkg/package.json')]: JSON.stringify({
37+
name: 'pkg',
38+
main: 'lib/index',
39+
browser: {'./lib/redirected.js': './lib/other.js'},
40+
}),
41+
[p('/root/node_modules/pkg/lib/index.js')]: '',
42+
[p('/root/node_modules/pkg/lib/sub.js')]: '',
43+
[p('/root/node_modules/pkg/lib/other.js')]: '',
44+
[p('/root/node_modules/nopkg/index.js')]: '',
45+
};
46+
47+
let getPackageForModule: JestMockFn<[string], ?PackageForModule>;
48+
let context: ResolutionContext;
49+
50+
function useFileMap(files: typeof fileMap) {
51+
const accessors = createPackageAccessors(files);
52+
getPackageForModule = jest.fn(accessors.getPackageForModule);
53+
context = {
54+
...createResolutionContext(files),
55+
...accessors,
56+
getPackageForModule,
57+
originModulePath: p('/root/src/main.js'),
58+
};
59+
}
60+
61+
beforeEach(() => {
62+
useFileMap(fileMap);
63+
});
64+
65+
const lookedUpPaths = () =>
66+
getPackageForModule.mock.calls.map(([modulePath]) => modulePath);
67+
68+
test('relative file, found after trying several extensions', () => {
69+
expect(Resolver.resolve(context, './foo', 'ios')).toEqual({
70+
type: 'sourceFile',
71+
filePath: p('/root/src/foo.js'),
72+
});
73+
expect(lookedUpPaths()).toEqual([p('/root/src/foo'), filesIn('/root/src')]);
74+
});
75+
76+
test('relative file, when the stand-in file name is a package root', () => {
77+
useFileMap({
78+
...fileMap,
79+
// A package whose "browser" field must not apply to the files beside it
80+
[p('/root/src/.metro-package-scope/package.json')]: JSON.stringify({
81+
name: 'stand-in',
82+
browser: {'./foo.js': false},
83+
}),
84+
});
85+
expect(Resolver.resolve(context, './foo', 'ios')).toEqual({
86+
type: 'sourceFile',
87+
filePath: p('/root/src/foo.js'),
88+
});
89+
// The stand-in is retried with another name
90+
expect(lookedUpPaths()).toEqual([
91+
p('/root/src/foo'),
92+
filesIn('/root/src'),
93+
filesIn('/root/src') + '_',
94+
]);
95+
});
96+
97+
test('throws if the scope is never that of the stand-in file', () => {
98+
getPackageForModule = jest.fn(() => ({
99+
rootPath: p('/root'),
100+
packageJson: {name: 'root'},
101+
packageRelativePath: '',
102+
}));
103+
context = {...context, getPackageForModule};
104+
expect(() => Resolver.resolve(context, './foo', 'ios')).toThrow(
105+
`getPackageForModule(${JSON.stringify(filesIn('/root/src') + '__')}) ` +
106+
'returned packageRelativePath "", which should end in ' +
107+
'".metro-package-scope__".',
108+
);
109+
expect(lookedUpPaths()).toEqual([
110+
p('/root/src/foo'),
111+
filesIn('/root/src'),
112+
filesIn('/root/src') + '_',
113+
filesIn('/root/src') + '__',
114+
]);
115+
});
116+
117+
test('relative file in the file system root', () => {
118+
useFileMap({[p('/main.js')]: '', [p('/foo.js')]: ''});
119+
context = {...context, originModulePath: p('/main.js')};
120+
expect(Resolver.resolve(context, './foo', 'ios')).toEqual({
121+
type: 'sourceFile',
122+
filePath: p('/foo.js'),
123+
});
124+
expect(lookedUpPaths()).toEqual([p('/foo'), filesIn('')]);
125+
});
126+
127+
test('relative file with an explicit extension', () => {
128+
expect(Resolver.resolve(context, './foo.js', 'ios')).toEqual({
129+
type: 'sourceFile',
130+
filePath: p('/root/src/foo.js'),
131+
});
132+
expect(lookedUpPaths()).toEqual([
133+
p('/root/src/foo.js'),
134+
filesIn('/root/src'),
135+
]);
136+
});
137+
138+
test('relative directory, resolved to its index', () => {
139+
expect(Resolver.resolve(context, './lib', 'ios')).toEqual({
140+
type: 'sourceFile',
141+
filePath: p('/root/src/lib/index.js'),
142+
});
143+
expect(lookedUpPaths()).toEqual([
144+
p('/root/src/lib'),
145+
filesIn('/root/src'),
146+
filesIn('/root/src/lib'),
147+
]);
148+
});
149+
150+
test('package entry point', () => {
151+
expect(Resolver.resolve(context, 'pkg', 'ios')).toEqual({
152+
type: 'sourceFile',
153+
filePath: p('/root/node_modules/pkg/lib/index.js'),
154+
});
155+
// The origin, the package directory, the files in node_modules (`pkg` as
156+
// a file), and the files in the directory of the entry point.
157+
expect(lookedUpPaths()).toEqual([
158+
p('/root/src/main.js'),
159+
p('/root/node_modules/pkg'),
160+
p('/root/node_modules/pkg'),
161+
filesIn('/root/node_modules'),
162+
filesIn('/root/node_modules/pkg/lib'),
163+
]);
164+
});
165+
166+
test('package subpath', () => {
167+
expect(Resolver.resolve(context, 'pkg/lib/sub', 'ios')).toEqual({
168+
type: 'sourceFile',
169+
filePath: p('/root/node_modules/pkg/lib/sub.js'),
170+
});
171+
expect(lookedUpPaths()).toEqual([
172+
p('/root/src/main.js'),
173+
p('/root/node_modules/pkg/lib/sub'),
174+
p('/root/node_modules/pkg/lib/sub'),
175+
filesIn('/root/node_modules/pkg/lib'),
176+
]);
177+
});
178+
179+
test('package subpath redirected by the "browser" field', () => {
180+
expect(Resolver.resolve(context, 'pkg/lib/redirected', 'ios')).toEqual({
181+
type: 'sourceFile',
182+
filePath: p('/root/node_modules/pkg/lib/other.js'),
183+
});
184+
// The redirected path may be redirected again by its own scope
185+
expect(lookedUpPaths()).toEqual([
186+
p('/root/src/main.js'),
187+
p('/root/node_modules/pkg/lib/redirected'),
188+
p('/root/node_modules/pkg/lib/other.js'),
189+
filesIn('/root/node_modules/pkg/lib'),
190+
]);
191+
});
192+
193+
test('module under node_modules without a package.json', () => {
194+
expect(Resolver.resolve(context, 'nopkg', 'ios')).toEqual({
195+
type: 'sourceFile',
196+
filePath: p('/root/node_modules/nopkg/index.js'),
197+
});
198+
expect(lookedUpPaths()).toEqual([
199+
p('/root/src/main.js'),
200+
p('/root/node_modules/nopkg'),
201+
p('/root/node_modules/nopkg'),
202+
filesIn('/root/node_modules'),
203+
filesIn('/root/node_modules/nopkg'),
204+
]);
205+
});
206+
207+
test('package subpath with package exports', () => {
208+
useFileMap({
209+
...fileMap,
210+
[p('/root/node_modules/pkg/package.json')]: JSON.stringify({
211+
name: 'pkg',
212+
exports: {'./sub': './lib/sub.js'},
213+
}),
214+
});
215+
context = {...context, unstable_enablePackageExports: true};
216+
expect(Resolver.resolve(context, 'pkg/sub', 'ios')).toEqual({
217+
type: 'sourceFile',
218+
filePath: p('/root/node_modules/pkg/lib/sub.js'),
219+
});
220+
expect(lookedUpPaths()).toEqual([
221+
p('/root/src/main.js'),
222+
p('/root/node_modules/pkg/sub'),
223+
p('/root/node_modules/pkg/sub'),
224+
]);
225+
});
226+
});

0 commit comments

Comments
 (0)