Repository navigation
Commit c5e27e6
committed
Resolver: Collect file system observations per resolution under unstable_incrementalResolution
Stacked on #1948.
The first consumer of the `Observations` API from #1944. When `resolver.unstable_incrementalResolution` is enabled, `ModuleResolver.resolveDependency` creates an observations record at the start of each resolution, and everything that resolution looks up in the file map is recorded into it. The record is returned on the `BundlerResolution` as `unstable_observations`. Nothing reads it yet - invalidating from it is the next step - so there's no behaviour change, and with the flag off nothing is allocated.
The resolution context given to `metro-resolver` doesn't change. `ModuleResolver` already builds a context per resolution, so when the flag is on it binds the file system capabilities to that resolution's record (`fileSystemLookup: p => fileSystemLookup(p, observations)`), and when it's off it passes the same functions as before. That means a custom `resolveRequest` records whatever it looks up through the context, without knowing about any of this. The observations parameter only appears on Metro's internal option types.
What's recorded:
- `fileSystemLookup`, and `resolveAsset` through it.
- `getPackageForModule`, down to `hierarchicalLookup`. There's no memo in the way since #1948.
- `doesFileExist`. `FileSystem.exists` can't record, so when recording this is answered by `lookup`, which agrees with it - true for exactly the paths found to be a file.
Putting the record on the `BundlerResolution` means `DependencyGraph`'s whole-resolution cache replays it for free - every origin given a cached resolution gets its observations too. The one other memo is the empty module, which is resolved once and cached, so a resolution that lands on it returns a copy with the union of what led there and what resolving the empty module observed.
Not included, to follow:
- The contents of `package.json`. `getPackage` reads with `fs` outside the file map, and Metro has no way to turn an absolute path into a canonical one. The file is recorded in `existence` by `hierarchicalLookup`, so adding or removing one is covered, but editing one isn't.
- Haste lookups, which have no bucket yet.
- Failed resolutions. They throw, so there's nowhere to put what they observed.
Resolution time in isolation, replaying every `DependencyGraph.resolveDependency` call made by one iOS dev build of the benchmark app (described in the test plan), through the whole-resolution cache. Nothing is transformed or serialised, so these are percentages of resolution, not of a build. 7 interleaved rounds. Resolutions are identical with the flag on and off.
| scenario | #1948 | this diff, flag off | this diff, flag on |
|---|---|---|---|
| first build (ms) | 95.9 | 95.8 (−0.1%) | 111.6 (+16.5%) |
| rebuild, nothing changed (ms) | 7.61 | 7.64 (+0.4%) | 7.80 (+2.1%) |
| after a source edit, re-resolving the edited module's dependencies (µs) | 153 | 159 (+4.1%) | 180 (+13.2%) |
| after a `package.json` change, re-resolving everything (ms) | 79.8 | 80.1 (+0.4%) | 93.1 (+16.2%) |
| heap retained after resolving (MB) | | 5.3 | 18.8 (3.5x) |
Flag off is within noise of #1948. (The source edit row is 8 resolutions, so that's under a microsecond each, and the runs overlap.)
Flag on costs about 16% of resolution time wherever everything is resolved, which is ~2µs per resolution, and 3.5x the retained heap. On the benchmark app that's 100k entries across 6.9k distinct resolutions - a median of 14 each, 90th percentile 27 - naming 28k distinct paths. Nearly all of it is `existence`, and most of that is extension and platform candidates that missed (`a.ios.js`, `a.native.js`, ...), which can't collapse because the directory they're in exists. That's the memory to go after before this is worth turning on, e.g. by recording the directory once instead.
```
yarn jest packages/metro-file-map packages/metro-resolver packages/metro/src/node-haste packages/metro/src/DeltaBundler packages/metro/src/integration_tests
yarn flow check
yarn typecheck-ts
yarn verify-api-snapshots
```
New tests in `resolver-test.js` assert the exact sets recorded for a relative import's candidates, each level probed for a package, a symlink (recorded as `content`, with lookups recorded by real path), asset variants, and a resolution that lands on the empty module. They also check that there's no `unstable_observations` by default, and that origins sharing a cached resolution share its observations.
**Benchmark app** - a synthetic mid-size RN 0.87 app: 23 common dependencies (Reanimated, React Navigation, TanStack Query, lodash, etc.) and ~1,000 generated first-party modules, 60% of them in a workspace package outside `projectRoot` with its own `node_modules`. The iOS dev bundle is 2.8k modules, from a file map of 34k files. Timings replay the `resolveDependency` calls a real build of it makes against a fresh `DependencyGraph` per process, in interleaved rounds, with source and `package.json` edits made on disk and picked up by the watcher. A real build of this app takes around 4s even with a warm transform cache, so resolution is about 2% of it.
Changelog: Internal1 parent 2568f43 commit c5e27e6
6 files changed
Lines changed: 326 additions & 26 deletions
File tree
- packages/metro/src
- DeltaBundler
- __tests__
- node-haste
- DependencyGraph
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2998 | 2998 | | |
2999 | 2999 | | |
3000 | 3000 | | |
| 3001 | + | |
| 3002 | + | |
| 3003 | + | |
| 3004 | + | |
| 3005 | + | |
| 3006 | + | |
| 3007 | + | |
| 3008 | + | |
| 3009 | + | |
| 3010 | + | |
| 3011 | + | |
| 3012 | + | |
| 3013 | + | |
| 3014 | + | |
| 3015 | + | |
| 3016 | + | |
| 3017 | + | |
| 3018 | + | |
| 3019 | + | |
| 3020 | + | |
| 3021 | + | |
| 3022 | + | |
| 3023 | + | |
| 3024 | + | |
| 3025 | + | |
| 3026 | + | |
| 3027 | + | |
| 3028 | + | |
| 3029 | + | |
| 3030 | + | |
| 3031 | + | |
| 3032 | + | |
| 3033 | + | |
| 3034 | + | |
| 3035 | + | |
| 3036 | + | |
| 3037 | + | |
| 3038 | + | |
| 3039 | + | |
| 3040 | + | |
| 3041 | + | |
| 3042 | + | |
| 3043 | + | |
| 3044 | + | |
| 3045 | + | |
| 3046 | + | |
| 3047 | + | |
| 3048 | + | |
| 3049 | + | |
| 3050 | + | |
| 3051 | + | |
| 3052 | + | |
| 3053 | + | |
| 3054 | + | |
| 3055 | + | |
| 3056 | + | |
| 3057 | + | |
| 3058 | + | |
| 3059 | + | |
| 3060 | + | |
| 3061 | + | |
| 3062 | + | |
| 3063 | + | |
| 3064 | + | |
| 3065 | + | |
| 3066 | + | |
| 3067 | + | |
| 3068 | + | |
| 3069 | + | |
| 3070 | + | |
| 3071 | + | |
| 3072 | + | |
| 3073 | + | |
| 3074 | + | |
| 3075 | + | |
| 3076 | + | |
| 3077 | + | |
| 3078 | + | |
| 3079 | + | |
| 3080 | + | |
| 3081 | + | |
| 3082 | + | |
| 3083 | + | |
| 3084 | + | |
| 3085 | + | |
| 3086 | + | |
| 3087 | + | |
| 3088 | + | |
| 3089 | + | |
| 3090 | + | |
| 3091 | + | |
| 3092 | + | |
| 3093 | + | |
| 3094 | + | |
| 3095 | + | |
| 3096 | + | |
| 3097 | + | |
| 3098 | + | |
| 3099 | + | |
| 3100 | + | |
| 3101 | + | |
| 3102 | + | |
| 3103 | + | |
| 3104 | + | |
| 3105 | + | |
| 3106 | + | |
| 3107 | + | |
| 3108 | + | |
| 3109 | + | |
| 3110 | + | |
| 3111 | + | |
| 3112 | + | |
| 3113 | + | |
| 3114 | + | |
| 3115 | + | |
| 3116 | + | |
| 3117 | + | |
| 3118 | + | |
| 3119 | + | |
| 3120 | + | |
| 3121 | + | |
| 3122 | + | |
| 3123 | + | |
| 3124 | + | |
| 3125 | + | |
| 3126 | + | |
| 3127 | + | |
| 3128 | + | |
| 3129 | + | |
| 3130 | + | |
| 3131 | + | |
| 3132 | + | |
| 3133 | + | |
| 3134 | + | |
| 3135 | + | |
| 3136 | + | |
| 3137 | + | |
| 3138 | + | |
| 3139 | + | |
| 3140 | + | |
| 3141 | + | |
| 3142 | + | |
| 3143 | + | |
| 3144 | + | |
| 3145 | + | |
| 3146 | + | |
| 3147 | + | |
| 3148 | + | |
| 3149 | + | |
| 3150 | + | |
| 3151 | + | |
| 3152 | + | |
| 3153 | + | |
| 3154 | + | |
| 3155 | + | |
| 3156 | + | |
| 3157 | + | |
| 3158 | + | |
| 3159 | + | |
| 3160 | + | |
| 3161 | + | |
| 3162 | + | |
| 3163 | + | |
| 3164 | + | |
| 3165 | + | |
| 3166 | + | |
| 3167 | + | |
| 3168 | + | |
| 3169 | + | |
| 3170 | + | |
| 3171 | + | |
| 3172 | + | |
| 3173 | + | |
| 3174 | + | |
| 3175 | + | |
| 3176 | + | |
| 3177 | + | |
| 3178 | + | |
| 3179 | + | |
3001 | 3180 | | |
3002 | 3181 | | |
3003 | 3182 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
151 | 151 | | |
152 | 152 | | |
153 | 153 | | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
154 | 162 | | |
155 | 163 | | |
156 | 164 | | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
157 | 169 | | |
158 | 170 | | |
159 | 171 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| 14 | + | |
14 | 15 | | |
15 | 16 | | |
16 | 17 | | |
| |||
135 | 136 | | |
136 | 137 | | |
137 | 138 | | |
138 | | - | |
139 | | - | |
| 139 | + | |
| 140 | + | |
140 | 141 | | |
141 | 142 | | |
142 | 143 | | |
| |||
193 | 194 | | |
194 | 195 | | |
195 | 196 | | |
| 197 | + | |
196 | 198 | | |
197 | 199 | | |
198 | 200 | | |
| |||
201 | 203 | | |
202 | 204 | | |
203 | 205 | | |
| 206 | + | |
204 | 207 | | |
205 | 208 | | |
206 | 209 | | |
| |||
0 commit comments