Skip to content

Commit 51d99a7

Browse files
committed
metro-file-map: Let hierarchicalLookup take a basename, checked with one map lookup
Stacked on #1945. ## Summary `hierarchicalLookup` walks up from a start path asking whether each ancestor contains a given subpath - it's how `DependencyGraph` finds the closest `package.json`. For every candidate ancestor it joined the candidate's path to the subpath and traversed the result from the root again, even though the traversal that collected the ancestors already holds each candidate's node. (The `start` argument the loop built for this was never read.) This lets the caller give either `opts.subpath`, which may be deep and behaves as before, or `opts.basename`, a single path segment. A basename is checked with one `Map.get` on the candidate's own node. It isn't validated to be a single segment - that'd be a cost on every call for something the caller already knows. With a basename, two cases still traverse: - A symlinked child, because we need to follow it to learn its type and real path. - Ancestors of the project root. Their nodes don't hold the segment that leads back down towards the root (with a root of `/A/B/C`, the node for `/A/B` has no `C` entry), so a map lookup there could wrongly report a directory as missing. Observations are recorded exactly as before. The existing test table now runs every row both as a `basename` and as a `subpath`, and the two must agree on the result and on the recorded sets. ### Compatibility The subpath moves from a positional argument into `opts`, so this changes the call shape for anything calling `hierarchicalLookup`. Metro's only caller passes `package.json`. Expo CLI's `createTypescriptResolver` passes deep subpaths (`node_modules/${targetPath}/tsconfig.json`) - that currently runs against `@expo/metro-file-map`'s own port of `TreeFS`, so it isn't affected, and deep subpaths remain supported here via `opts.subpath`. ## Benchmark Same harness as #1944 (6,935 resolutions, 7 interleaved rounds). Resolver output is byte-identical. | metric | #1945 | this diff | |---|---|---| | first pass after startup (ms) | 85.9 | **81.8 (−4.8%)** | | warm pass (ms) | 54.0 | 54.0 | | after a source change (ms) | 61.6 | 61.8 (+0.3%) | | after a `package.json` change (ms) | 64.4 | 63.7 (−1.1%) | Only the first pass moves, which is expected - after that these lookups are served by `PackageCache`. Isolating the walks themselves, by replaying the 4,483 closest-package walks of one pass directly against `hierarchicalLookup` (9 interleaved rounds): | | per walk | | |---|---|---| | #1945 | 865ns | | | this diff, `basename: 'package.json'` | **387ns** | **−55%** | | alternative: API unchanged, take the fast path when `subpath` has no separator | 394ns | −54% | The alternative is within noise of this, so the choice between them is about the API rather than speed. A deep subpath (`node_modules/react/package.json` from the same start paths) is 1,083ns -> 1,120ns per walk with either, about 3%, for the extra branch per candidate. ## Test plan ``` yarn jest packages/metro-file-map packages/metro-resolver packages/metro/src/node-haste packages/metro/src/DeltaBundler/__tests__/resolver-test.js yarn flow check yarn typecheck-ts yarn verify-api-snapshots ``` New tests cover a directory-type lookup by basename, and a deep subpath. The existing fixture already covers a symlinked `package.json`, a broken link, and a directory named `package.json`. Changelog: Internal
1 parent b28de70 commit 51d99a7

5 files changed

Lines changed: 158 additions & 69 deletions

File tree

‎packages/metro-file-map/API.md‎

Lines changed: 1 addition & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -142,15 +142,7 @@ interface FileSystem_2 {
142142
getOrComputeSha1(file: Path): Promise<null | undefined | {sha1: string; content?: Buffer | undefined}>;
143143
getSerializableSnapshot(): CacheData['fileSystemData'];
144144
getSha1(file: Path): null | undefined | string;
145-
hierarchicalLookup(
146-
mixedStartPath: string,
147-
subpath: string,
148-
opts: {
149-
breakOnSegment?: null | undefined | string;
150-
subpathType: 'f' | 'd';
151-
},
152-
observations?: null | undefined | Observations,
153-
): null | undefined | {absolutePath: string; containerRelativePath: string};
145+
hierarchicalLookup(mixedStartPath: string, opts: HierarchicalLookupOptions, observations?: null | undefined | Observations): null | undefined | {absolutePath: string; containerRelativePath: string};
154146
linkStats(file: Path): null | undefined | FileStats;
155147
lookup(mixedPath: Path, observations?: null | undefined | Observations): LookupResult;
156148
matchFiles(opts: {

‎packages/metro-file-map/src/flow-types.js‎

Lines changed: 22 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -346,19 +346,20 @@ export interface FileSystem {
346346
* X = dirname(X)
347347
* while X !== dirname(X)
348348
*
349+
* The subpath is given as either `opts.subpath`, which may be deep, or
350+
* `opts.basename`, which must be a single path segment and is not checked
351+
* to be one. Prefer `basename` where it applies: each candidate is then
352+
* checked with one map lookup rather than a traversal.
353+
*
349354
* If `observations` is given, records the paths this result depends upon.
350355
*
351-
* Useful for finding the closest package scope (subpath: package.json,
356+
* Useful for finding the closest package scope (basename: package.json,
352357
* type f, breakOnSegment: node_modules) or closest potential package root
353358
* (subpath: node_modules/pkg, type: d) in Node.js resolution.
354359
*/
355360
hierarchicalLookup(
356361
mixedStartPath: string,
357-
subpath: string,
358-
opts: {
359-
breakOnSegment?: ?string,
360-
subpathType: 'f' | 'd',
361-
},
362+
opts: HierarchicalLookupOptions,
362363
observations?: ?Observations,
363364
): ?{
364365
absolutePath: string,
@@ -407,6 +408,21 @@ export type JsonData =
407408
| Array<JsonData>
408409
| {[key: string]: JsonData};
409410

411+
export type HierarchicalLookupOptions = Readonly<
412+
| {
413+
basename: string,
414+
subpath?: void,
415+
breakOnSegment?: ?string,
416+
subpathType: 'f' | 'd',
417+
}
418+
| {
419+
basename?: void,
420+
subpath: string,
421+
breakOnSegment?: ?string,
422+
subpathType: 'f' | 'd',
423+
},
424+
>;
425+
410426
export type LookupResult =
411427
| {
412428
// The node is missing from the FileSystem implementation (note this

‎packages/metro-file-map/src/lib/TreeFS.js‎

Lines changed: 57 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ import type {
1414
FileMetadata,
1515
FileStats,
1616
FileSystemListener,
17+
HierarchicalLookupOptions,
1718
LookupResult,
1819
MutableFileSystem,
1920
Observations,
@@ -845,25 +846,29 @@ export default class TreeFS implements MutableFileSystem {
845846
* X = dirname(X)
846847
* while X !== dirname(X)
847848
*
849+
* The subpath is given as either `opts.subpath`, which may be deep, or
850+
* `opts.basename`, which must be a single path segment and is not checked
851+
* to be one. Prefer `basename` where it applies: each candidate is then
852+
* checked with one map lookup rather than a traversal.
853+
*
848854
* If `observations` is given, records the canonical paths this result
849855
* depends upon.
850856
*
851-
* Useful for finding the closest package scope (subpath: package.json,
857+
* Useful for finding the closest package scope (basename: package.json,
852858
* type f, breakOnSegment: node_modules) or closest potential package root
853859
* (subpath: node_modules/pkg, type: d) in Node.js resolution.
854860
*/
855861
hierarchicalLookup(
856862
mixedStartPath: string,
857-
subpath: string,
858-
opts: {
859-
breakOnSegment: ?string,
860-
subpathType: 'f' | 'd',
861-
},
863+
opts: HierarchicalLookupOptions,
862864
observations?: ?Observations,
863865
): ?{
864866
absolutePath: string,
865867
containerRelativePath: string,
866868
} {
869+
// Only a basename can be looked up on a candidate's own node.
870+
const isBasename = opts.basename != null;
871+
const subpath: string = opts.basename ?? opts.subpath ?? '';
867872
const ancestorsOfInput: Array<{
868873
ancestorOfRootIdx: ?number,
869874
node: DirectoryNode,
@@ -882,7 +887,11 @@ export default class TreeFS implements MutableFileSystem {
882887
subpath,
883888
opts.subpathType,
884889
observations,
885-
null,
890+
isBasename &&
891+
(closestLookup.ancestorOfRootIdx == null ||
892+
closestLookup.ancestorOfRootIdx === 0)
893+
? closestLookup.node
894+
: null,
886895
);
887896
if (maybeAbsolutePathMatch != null) {
888897
return {
@@ -955,14 +964,7 @@ export default class TreeFS implements MutableFileSystem {
955964
subpath,
956965
opts.subpathType,
957966
observations,
958-
{
959-
ancestorOfRootIdx: candidate.ancestorOfRootIdx,
960-
node: candidate.node,
961-
pathIdx:
962-
candidate.normalPath.length > 0
963-
? candidate.normalPath.length + 1
964-
: 0,
965-
},
967+
isBasename ? candidate.node : null,
966968
);
967969
if (maybeAbsolutePathMatch != null) {
968970
// Determine the input path relative to the current candidate. Note
@@ -1002,7 +1004,9 @@ export default class TreeFS implements MutableFileSystem {
10021004
subpath,
10031005
opts.subpathType,
10041006
observations,
1005-
null,
1007+
isBasename && depthBelowCommonRoot === 0 && commonRootDepth === 0
1008+
? nextNode
1009+
: null,
10061010
);
10071011
if (maybeAbsolutePathMatch != null) {
10081012
const rootDirParts = this.#pathUtils.getParts();
@@ -1031,17 +1035,49 @@ export default class TreeFS implements MutableFileSystem {
10311035
return null;
10321036
}
10331037

1038+
/**
1039+
* Whether the real directory at `normalCandidatePath` has `subpath` of the
1040+
* given type, returning the absolute real path of the match.
1041+
*
1042+
* `candidateNode` is given when `subpath` is a basename and the candidate is
1043+
* the root or a descendant of it, and the child is then checked with one map
1044+
* lookup. It is omitted for an ancestor of the root, whose node does not
1045+
* hold the segment leading back towards the root, and for a deep subpath.
1046+
* Those are found by traversal instead.
1047+
*/
10341048
#checkCandidateHasSubpath(
10351049
normalCandidatePath: string,
10361050
subpath: string,
10371051
subpathType: 'f' | 'd',
10381052
observations: ?Observations,
1039-
start: ?{
1040-
ancestorOfRootIdx: ?number,
1041-
node: DirectoryNode,
1042-
pathIdx: number,
1043-
},
1053+
candidateNode: ?DirectoryNode,
10441054
): ?string {
1055+
if (candidateNode != null) {
1056+
const childNode = candidateNode.get(subpath);
1057+
// A symlink must be traversed to learn its type and real path, so only
1058+
// a missing child, a directory or a regular file is answered here.
1059+
if (
1060+
childNode == null ||
1061+
isDirectory(childNode) ||
1062+
isRegularFile(childNode)
1063+
) {
1064+
const isMatch =
1065+
childNode != null && isDirectory(childNode) === (subpathType === 'd');
1066+
if (!isMatch && observations == null) {
1067+
return null;
1068+
}
1069+
const childNormalPath =
1070+
normalCandidatePath === ''
1071+
? subpath
1072+
: normalCandidatePath + path.sep + subpath;
1073+
if (observations) {
1074+
observations.existence.add(childNormalPath);
1075+
}
1076+
return isMatch
1077+
? this.#pathUtils.normalToAbsolute(childNormalPath)
1078+
: null;
1079+
}
1080+
}
10451081
const lookupResult = this.#lookupByNormalPath(
10461082
this.#pathUtils.joinNormalToRelative(normalCandidatePath, subpath)
10471083
.normalPath,

‎packages/metro-file-map/src/lib/__tests__/TreeFS-test.js‎

Lines changed: 71 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -471,6 +471,46 @@ describe.each([['win32'], ['posix']])('TreeFS on %s', platform => {
471471
});
472472
});
473473

474+
test('finds the closest directory with the given basename', () => {
475+
const observations = emptyObservations();
476+
expect(
477+
tfs.hierarchicalLookup(
478+
p('/A/B/C/a/n_m/pkg/subpath/deep'),
479+
{basename: 'n_m', breakOnSegment: null, subpathType: 'd'},
480+
observations,
481+
),
482+
).toEqual({
483+
absolutePath: p('/A/B/C/a/n_m/pkg/n_m'),
484+
containerRelativePath: p('subpath/deep'),
485+
});
486+
expect(observations).toEqual({
487+
content: new Set(),
488+
existence: new Set([
489+
p('a/n_m/pkg/subpath/deep/n_m'),
490+
p('a/n_m/pkg/subpath/n_m'),
491+
p('a/n_m/pkg/n_m'),
492+
]),
493+
});
494+
});
495+
496+
test('finds the closest match for a deep subpath', () => {
497+
const observations = emptyObservations();
498+
expect(
499+
tfs.hierarchicalLookup(
500+
p('/A/B/C/a/b/c'),
501+
{subpath: p('n_m/pkg'), breakOnSegment: null, subpathType: 'd'},
502+
observations,
503+
),
504+
).toEqual({
505+
absolutePath: p('/A/B/C/a/n_m/pkg'),
506+
containerRelativePath: p('b/c'),
507+
});
508+
expect(observations).toEqual({
509+
content: new Set(),
510+
existence: new Set([p('a/b/c/n_m'), p('a/b/n_m'), p('a/n_m/pkg')]),
511+
});
512+
});
513+
474514
// The fourth column is paths, besides the match itself, whose addition or
475515
// removal changes the result, the fifth those whose modification does too -
476516
// the symlinks traversed.
@@ -605,36 +645,39 @@ describe.each([['win32'], ['posix']])('TreeFS on %s', platform => {
605645
) => {
606646
const pathMap = (normalPosixPath: string) =>
607647
mockPathModule.resolve(p('/A/B/C'), p(normalPosixPath));
608-
const observations = emptyObservations();
609-
expect(
610-
tfs.hierarchicalLookup(
611-
p(startPath),
612-
'package.json',
613-
{
614-
breakOnSegment: 'n_m',
615-
subpathType: 'f',
616-
},
617-
observations,
618-
),
619-
).toEqual(
620-
expectedPath == null
621-
? null
622-
: {
623-
absolutePath: pathMap(expectedPath),
624-
containerRelativePath: p(expectedRelativeSubpath),
625-
},
626-
);
627648
const toCanonical = (posixPath: string) =>
628649
canonicalTo(p('/A/B/C'))(pathMap(posixPath));
629-
expect(observations).toEqual({
630-
content: new Set(expectedContent.map(toCanonical)),
631-
existence: new Set(
632-
(expectedPath == null
633-
? expectedExistence
634-
: [...expectedExistence, expectedPath]
635-
).map(toCanonical),
636-
),
637-
});
650+
// A basename is looked up on each candidate's node and a subpath by
651+
// traversal, and the two must agree.
652+
for (const target of [
653+
{basename: 'package.json'},
654+
{subpath: 'package.json'},
655+
]) {
656+
const observations = emptyObservations();
657+
expect(
658+
tfs.hierarchicalLookup(
659+
p(startPath),
660+
{...target, breakOnSegment: 'n_m', subpathType: 'f'},
661+
observations,
662+
),
663+
).toEqual(
664+
expectedPath == null
665+
? null
666+
: {
667+
absolutePath: pathMap(expectedPath),
668+
containerRelativePath: p(expectedRelativeSubpath),
669+
},
670+
);
671+
expect(observations).toEqual({
672+
content: new Set(expectedContent.map(toCanonical)),
673+
existence: new Set(
674+
(expectedPath == null
675+
? expectedExistence
676+
: [...expectedExistence, expectedPath]
677+
).map(toCanonical),
678+
),
679+
});
680+
}
638681
},
639682
);
640683
});

‎packages/metro/src/node-haste/DependencyGraph.js‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,12 @@ const {createActionStartEntry, createActionEndEntry, log} = Logger;
4545

4646
const NULL_PLATFORM = Symbol();
4747

48+
const CLOSEST_PACKAGE_LOOKUP = {
49+
basename: 'package.json',
50+
breakOnSegment: 'node_modules',
51+
subpathType: 'f',
52+
} as const;
53+
4854
function getOrCreateMap<T>(
4955
map: Map<string | symbol, Map<string | symbol, T>>,
5056
field: string,
@@ -178,11 +184,7 @@ export default class DependencyGraph extends EventEmitter {
178184
): ?{packageJsonPath: string, packageRelativePath: string} {
179185
const result = this._fileSystem.hierarchicalLookup(
180186
absoluteModulePath,
181-
'package.json',
182-
{
183-
breakOnSegment: 'node_modules',
184-
subpathType: 'f',
185-
},
187+
CLOSEST_PACKAGE_LOOKUP,
186188
);
187189
return result
188190
? {

0 commit comments

Comments
 (0)