Skip to content

Commit 19f8d76

Browse files
committed
Resolver: Resolve the package of a directory once for all extension candidates
Stacked on #1944. ## Summary `resolveFile` tries each candidate extension in turn, and for each one `resolveSourceFileForExt` calls `redirectModulePath`, which asks `context.getPackageForModule` for the package scope of the candidate path. Every candidate in a directory belongs to the same package, so that's the same question asked once per extension - and because Metro's `PackageCache` memoises by module path, each distinct candidate string misses the first time it's seen, mostly for files that don't exist. This asks for the package of the directory once, up front, and shares it with every candidate via the source file context. `redirectPackageSubpath` is the part of `redirectModulePath` that applies `mainFields` redirections when the containing package is already known. `redirectModulePath` delegates to it, and behaves as before for its other callers. One wrinkle: files directly inside a `node_modules` directory belong to no package, but the package of that directory itself is whatever contains it, so `getPackageForFilesIn` handles that case explicitly. `package-scope-test.js` pins the sequence of `getPackageForModule` calls the resolver makes for the common specifier shapes, so that the call count is a tested property rather than incidental. This also removes the `@deprecated` tag from `getPackageForModule` - the resolver depends on it, and its doc now says what it returns for symlinked and missing paths. ### `resetCache` in tests Four tests in `resolver-test.js` build a second `DependencyGraph` on the same config, and the second reads the first's file map cache. The integration tests do the same across test files and across runs, because their file map cache defaults to `os.tmpdir()`. Under Jest a cached tree comes back from `v8.deserialize` with `Map`s from another realm, which fail `TreeFS`'s `instanceof Map` checks. Asking for the package of a directory reaches `hierarchicalLookup`'s root-ancestor invariant, which turns what was previously a silent wrong answer into a failure, so both configs set `resetCache: true`. #1934 fixes the underlying problem - once that lands, the line in `resolver-test.js` can go. The one in the integration config is worth keeping regardless, since those tests shouldn't depend on what a previous run left in the temp directory. ## Benchmark This cuts package scope lookups 3.7x, and takes 28% off first-pass resolution time, on our benchmark app (described in the test plan). 7 interleaved rounds. Resolver output is byte-identical. | metric | #1944 | this diff | |---|---|---| | `getPackageForModule` calls per pass | 64k | **17k (3.7x fewer)** | | first pass after startup (ms) | 119.6 | **85.7 (−28.3%)** | | warm pass (ms) | 55.2 | **53.5 (−3.1%)** | | after a source change (ms) | 63.0 | **60.9 (−3.4%)** | | after a `package.json` change (ms) | 68.4 | **64.0 (−6.4%)** | The first pass is where this lands, because that's where every distinct candidate path is a `PackageCache` miss and a walk up the tree. ## Test plan ``` yarn jest packages/metro-resolver packages/metro-file-map packages/metro/src/node-haste packages/metro/src/DeltaBundler/__tests__/resolver-test.js yarn flow check yarn verify-api-snapshots ``` **Benchmark app** - a synthetic mid-size RN 0.87 app: 23 common dependencies (Reanimated, React Navigation, TanStack Query, lodash, etc.) and ~1,000 generated first-party modules, 60% of them in a workspace package outside `projectRoot` with its own `node_modules`. The iOS dev bundle is 2.8k modules, from a file map of 34k files. Timings replay the resolutions a real build of it makes against a fresh `DependencyGraph` per process, in interleaved rounds. Changelog: Internal
1 parent 6c73d97 commit 19f8d76

6 files changed

Lines changed: 270 additions & 28 deletions

File tree

‎packages/metro-resolver/src/PackageResolve.js‎

Lines changed: 51 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -106,8 +106,6 @@ export function redirectModulePath(
106106
return modulePath;
107107
}
108108

109-
let redirectedPath;
110-
111109
if (modulePath.startsWith('.') || isModulePathAbsolute) {
112110
const packageRelativeModulePath = isModulePathAbsolute
113111
? // If the module path is absolute, containingPackage is relative to it
@@ -120,40 +118,68 @@ export function redirectModulePath(
120118
path.dirname(containingPackage.packageRelativePath),
121119
modulePath,
122120
);
123-
124-
redirectedPath = matchSubpathFromMainFields(
125-
// Use prefixed POSIX path for lookup in package.json
126-
'./' + systemToPosixPath(packageRelativeModulePath),
127-
containingPackage.packageJson,
128-
mainFields,
129-
);
130-
131-
if (typeof redirectedPath === 'string') {
132-
// BRITTLE ASSUMPTION: This is always treated as a package-relative path
133-
// and is converted back, even if the redirected path is a specifier
134-
// referring to another package.
135-
redirectedPath = path.isAbsolute(redirectedPath)
136-
? path.normalize(redirectedPath)
137-
: path.join(containingPackage.rootPath, redirectedPath);
138-
}
139-
} else {
140-
// Otherwise, `modulePath` may be an unprefixed relative path or a bare
141-
// specifier (can also be an absolute specifier prefixed with a URL scheme).
142-
// This is used only by the "browser" spec.
143-
redirectedPath = matchSubpathFromMainFields(
121+
return redirectPackageSubpath(
122+
context,
144123
modulePath,
145-
containingPackage.packageJson,
146-
mainFields,
124+
containingPackage,
125+
packageRelativeModulePath,
147126
);
148127
}
149128

129+
// Otherwise, `modulePath` may be an unprefixed relative path or a bare
130+
// specifier (can also be an absolute specifier prefixed with a URL scheme).
131+
// This is used only by the "browser" spec.
132+
const redirectedPath = matchSubpathFromMainFields(
133+
modulePath,
134+
containingPackage.packageJson,
135+
mainFields,
136+
);
137+
150138
if (redirectedPath != null) {
151139
return redirectedPath;
152140
}
153141

154142
return modulePath;
155143
}
156144

145+
/**
146+
* `redirectModulePath` for a module path whose containing package is already
147+
* known. `packageRelativePath` is the path of the module relative to the
148+
* package root, with system separators. Returns `false` if the module should
149+
* be ignored, and `modulePath` if no `package.json` mapping is matched.
150+
*/
151+
export function redirectPackageSubpath(
152+
context: Readonly<{
153+
mainFields: ResolutionContext['mainFields'],
154+
...
155+
}>,
156+
modulePath: string,
157+
containingPackage: Readonly<{
158+
packageJson: PackageInfo['packageJson'],
159+
rootPath: PackageInfo['rootPath'],
160+
...
161+
}>,
162+
packageRelativePath: string,
163+
): string | false {
164+
const redirectedPath = matchSubpathFromMainFields(
165+
// Use prefixed POSIX path for lookup in package.json
166+
'./' + systemToPosixPath(packageRelativePath),
167+
containingPackage.packageJson,
168+
context.mainFields,
169+
);
170+
171+
if (typeof redirectedPath === 'string') {
172+
// BRITTLE ASSUMPTION: This is always treated as a package-relative path
173+
// and is converted back, even if the redirected path is a specifier
174+
// referring to another package.
175+
return path.isAbsolute(redirectedPath)
176+
? path.normalize(redirectedPath)
177+
: path.join(containingPackage.rootPath, redirectedPath);
178+
}
179+
180+
return redirectedPath === false ? false : modulePath;
181+
}
182+
157183
/**
158184
* Get the mapped replacement for the given subpath defined by matching
159185
* `mainFields` entries in the passed `package.json`
Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
/**
2+
* Copyright (c) Meta Platforms, Inc. and affiliates.
3+
*
4+
* This source code is licensed under the MIT license found in the
5+
* LICENSE file in the root directory of this source tree.
6+
*
7+
* @flow strict-local
8+
* @format
9+
* @oncall react_native
10+
*/
11+
12+
import type {PackageForModule, ResolutionContext} from '../types';
13+
14+
import * as Resolver from '../index';
15+
import {createPackageAccessors, createResolutionContext} from './utils';
16+
17+
// The resolver looks up package scopes through `getPackageForModule`. These
18+
// tests pin down which paths it asks about for each kind of specifier, so
19+
// that changes to the number of lookups per resolution are deliberate. Metro
20+
// memoizes the answer per path, so a repeated path is cheap and a new one is
21+
// not: in particular, the directory of a file is looked up once for all of
22+
// its extension candidates.
23+
describe('package scope lookups', () => {
24+
const fileMap = {
25+
'/root/package.json': JSON.stringify({name: 'root'}),
26+
'/root/src/main.js': '',
27+
'/root/src/foo.js': '',
28+
'/root/src/lib/index.js': '',
29+
'/root/node_modules/pkg/package.json': JSON.stringify({
30+
name: 'pkg',
31+
main: 'lib/index',
32+
browser: {'./lib/redirected.js': './lib/other.js'},
33+
}),
34+
'/root/node_modules/pkg/lib/index.js': '',
35+
'/root/node_modules/pkg/lib/sub.js': '',
36+
'/root/node_modules/pkg/lib/other.js': '',
37+
'/root/node_modules/nopkg/index.js': '',
38+
};
39+
40+
let getPackageForModule: JestMockFn<[string], ?PackageForModule>;
41+
let context: ResolutionContext;
42+
43+
function useFileMap(files: typeof fileMap) {
44+
const accessors = createPackageAccessors(files);
45+
getPackageForModule = jest.fn(accessors.getPackageForModule);
46+
context = {
47+
...createResolutionContext(files),
48+
...accessors,
49+
getPackageForModule,
50+
originModulePath: '/root/src/main.js',
51+
};
52+
}
53+
54+
beforeEach(() => {
55+
useFileMap(fileMap);
56+
});
57+
58+
const lookedUpPaths = () =>
59+
getPackageForModule.mock.calls.map(([modulePath]) => modulePath);
60+
61+
test('relative file, found after trying several extensions', () => {
62+
expect(Resolver.resolve(context, './foo', 'ios')).toEqual({
63+
type: 'sourceFile',
64+
filePath: '/root/src/foo.js',
65+
});
66+
expect(lookedUpPaths()).toEqual(['/root/src/foo', '/root/src']);
67+
});
68+
69+
test('relative file with an explicit extension', () => {
70+
expect(Resolver.resolve(context, './foo.js', 'ios')).toEqual({
71+
type: 'sourceFile',
72+
filePath: '/root/src/foo.js',
73+
});
74+
expect(lookedUpPaths()).toEqual(['/root/src/foo.js', '/root/src']);
75+
});
76+
77+
test('relative directory, resolved to its index', () => {
78+
expect(Resolver.resolve(context, './lib', 'ios')).toEqual({
79+
type: 'sourceFile',
80+
filePath: '/root/src/lib/index.js',
81+
});
82+
expect(lookedUpPaths()).toEqual([
83+
'/root/src/lib',
84+
'/root/src',
85+
'/root/src/lib',
86+
]);
87+
});
88+
89+
test('package entry point', () => {
90+
expect(Resolver.resolve(context, 'pkg', 'ios')).toEqual({
91+
type: 'sourceFile',
92+
filePath: '/root/node_modules/pkg/lib/index.js',
93+
});
94+
// The origin, the package directory, and the directory of the entry
95+
// point. Files directly under node_modules have no scope, so none is
96+
// looked up for the `pkg` file candidates.
97+
expect(lookedUpPaths()).toEqual([
98+
'/root/src/main.js',
99+
'/root/node_modules/pkg',
100+
'/root/node_modules/pkg',
101+
'/root/node_modules/pkg/lib',
102+
]);
103+
});
104+
105+
test('package subpath', () => {
106+
expect(Resolver.resolve(context, 'pkg/lib/sub', 'ios')).toEqual({
107+
type: 'sourceFile',
108+
filePath: '/root/node_modules/pkg/lib/sub.js',
109+
});
110+
expect(lookedUpPaths()).toEqual([
111+
'/root/src/main.js',
112+
'/root/node_modules/pkg/lib/sub',
113+
'/root/node_modules/pkg/lib/sub',
114+
'/root/node_modules/pkg/lib',
115+
]);
116+
});
117+
118+
test('package subpath redirected by the "browser" field', () => {
119+
expect(Resolver.resolve(context, 'pkg/lib/redirected', 'ios')).toEqual({
120+
type: 'sourceFile',
121+
filePath: '/root/node_modules/pkg/lib/other.js',
122+
});
123+
// The redirected path may be redirected again by its own scope
124+
expect(lookedUpPaths()).toEqual([
125+
'/root/src/main.js',
126+
'/root/node_modules/pkg/lib/redirected',
127+
'/root/node_modules/pkg/lib/other.js',
128+
'/root/node_modules/pkg/lib',
129+
]);
130+
});
131+
132+
test('module under node_modules without a package.json', () => {
133+
expect(Resolver.resolve(context, 'nopkg', 'ios')).toEqual({
134+
type: 'sourceFile',
135+
filePath: '/root/node_modules/nopkg/index.js',
136+
});
137+
expect(lookedUpPaths()).toEqual([
138+
'/root/src/main.js',
139+
'/root/node_modules/nopkg',
140+
'/root/node_modules/nopkg',
141+
'/root/node_modules/nopkg',
142+
]);
143+
});
144+
145+
test('package subpath with package exports', () => {
146+
useFileMap({
147+
...fileMap,
148+
'/root/node_modules/pkg/package.json': JSON.stringify({
149+
name: 'pkg',
150+
exports: {'./sub': './lib/sub.js'},
151+
}),
152+
});
153+
context = {...context, unstable_enablePackageExports: true};
154+
expect(Resolver.resolve(context, 'pkg/sub', 'ios')).toEqual({
155+
type: 'sourceFile',
156+
filePath: '/root/node_modules/pkg/lib/sub.js',
157+
});
158+
expect(lookedUpPaths()).toEqual([
159+
'/root/src/main.js',
160+
'/root/node_modules/pkg/sub',
161+
'/root/node_modules/pkg/sub',
162+
]);
163+
});
164+
});

‎packages/metro-resolver/src/resolve.js‎

Lines changed: 39 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@
1212
import type {
1313
FileAndDirCandidates,
1414
FileCandidates,
15+
PackageForModule,
1516
Resolution,
1617
ResolutionContext,
1718
Result,
@@ -31,6 +32,7 @@ import {
3132
getPackageEntryPoint,
3233
matchSubpathFromMainFields,
3334
redirectModulePath,
35+
redirectPackageSubpath,
3436
} from './PackageResolve';
3537
import resolveAsset from './resolveAsset';
3638
import isAssetFile from './utils/isAssetFile';
@@ -743,7 +745,13 @@ function resolveFile(
743745

744746
const candidateExts: Array<string> = [];
745747
const filePathPrefix = path.join(dirPath, fileName);
746-
const sfContext = {...context, candidateExts, filePathPrefix};
748+
const sfContext = {
749+
...context,
750+
candidateExts,
751+
filePathPrefix,
752+
fileName,
753+
dirPackage: getPackageForFilesIn(context, dirPath),
754+
};
747755
const sourceFileResolution = resolveSourceFile(sfContext, platform);
748756
if (sourceFileResolution != null) {
749757
if (typeof sourceFileResolution === 'string') {
@@ -758,6 +766,10 @@ type SourceFileContext = Readonly<{
758766
...ResolutionContext,
759767
candidateExts: Array<string>,
760768
filePathPrefix: string,
769+
fileName: string,
770+
// The package containing the files in the directory, shared by every
771+
// candidate extension
772+
dirPackage: ?PackageForModule,
761773
}>;
762774

763775
// Either a full path, or a restricted subset of Resolution.
@@ -829,9 +841,20 @@ function resolveSourceFileForExt(
829841
extension: string,
830842
): SourceFileResolution {
831843
const filePath = `${context.filePathPrefix}${extension}`;
844+
const {dirPackage} = context;
832845
const redirectedPath =
833846
// Any redirections for the bare path have already happened
834-
extension !== '' ? redirectModulePath(context, filePath) : filePath;
847+
extension !== '' && dirPackage != null
848+
? redirectPackageSubpath(
849+
context,
850+
filePath,
851+
dirPackage,
852+
path.join(
853+
dirPackage.packageRelativePath,
854+
context.fileName + extension,
855+
),
856+
)
857+
: filePath;
835858
if (redirectedPath === false) {
836859
return {type: 'empty'};
837860
}
@@ -843,6 +866,20 @@ function resolveSourceFileForExt(
843866
return null;
844867
}
845868

869+
/**
870+
* The package containing the files in a directory, which need not exist. This
871+
* is the package of the directory itself, except that the files directly in a
872+
* `node_modules` directory belong to none.
873+
*/
874+
function getPackageForFilesIn(
875+
context: ResolutionContext,
876+
dirPath: string,
877+
): ?PackageForModule {
878+
return dirPath.endsWith(path.sep + 'node_modules')
879+
? null
880+
: context.getPackageForModule(dirPath);
881+
}
882+
846883
function isRelativeImport(filePath: string) {
847884
return /^[.][.]?(?:[/]|$)/.test(filePath);
848885
}

‎packages/metro-resolver/src/types.js‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -167,7 +167,12 @@ export type ResolutionContext = Readonly<{
167167
* for a given absolute candidate path (which need not exist), or null if
168168
* there is no package.json closer than the nearest node_modules directory.
169169
*
170-
* @deprecated See https://github.com/react/metro/commit/29c77bff31e2475a086bc3f04073f485da8f9ff0
170+
* The scope is that of the real path: a path through a symlinked directory
171+
* belongs to the package containing the link target. A path that does not
172+
* exist belongs to the package of its deepest existing ancestor, unless the
173+
* first missing segment is `node_modules`.
174+
*
175+
* The resolver calls this a few times per resolution, so it must be cheap.
171176
*/
172177
getPackageForModule: (absoluteModulePath: string) => ?PackageForModule,
173178

‎packages/metro/src/DeltaBundler/__tests__/resolver-test.js‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -110,6 +110,10 @@ function dep(name: string): TransformResultDependency {
110110
maxWorkers: 1,
111111
projectRoot: p('/root'),
112112
reporter: require('../../lib/reporting').nullReporter,
113+
// Tests that create a second resolver must not read the first's file map
114+
// cache: `v8.deserialize` runs outside the Jest sandbox, so a cached tree
115+
// holds `Map`s from another realm that fail TreeFS's `instanceof` checks.
116+
resetCache: true,
113117
transformer: {},
114118
watchFolders: [p('/root')],
115119
};

‎packages/metro/src/integration_tests/metro.config.js‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,12 @@ module.exports = {
1919
maxWorkers: 1,
2020
projectRoot: ROOT_PATH,
2121
reporter: {update() {}},
22+
// The file map cache defaults to `os.tmpdir()` and is keyed only by build
23+
// parameters, so without this every test file, and every run, reads whatever
24+
// the previous one wrote. A cache read inside Jest also holds `Map`s from
25+
// another realm (`v8.deserialize` runs outside the sandbox) that fail
26+
// TreeFS's `instanceof` checks.
27+
resetCache: true,
2228
watchFolders: [path.resolve(__dirname, '../../../')],
2329
server: {port: 0},
2430
resolver: {

0 commit comments

Comments
 (0)