Skip to content

Commit 6366581

Browse files
GijsWeteringsfacebook-github-bot
authored andcommitted
Remote cache: fix write-back during hedged reads and a getParams edge case (#2251)
Summary: Follow-ups from the react/metro review of the two remote cache changes (#2249, #2246). **A hedged local transform no longer writes to a remote store that is about to hit.** `Cache.set` skips the store that served a hit and the stores after it, using `Cache.#hits`, which is only filled once a `get` returns. With `unstable_hedgeCacheReadsAfterMs`, the local transform can win while the remote read is still pending, so `set` wrote to every store: with a read-write remote (`HttpStore`), one redundant upload per file that already exists remotely, on the congested link the option is for. `HttpGetStore` does not write, so the setup this was built for was not affected. `Cache` now tracks the store each pending `get` is reading. A `set` during that read writes the stores the read has already missed at once, and the rest when the read settles, stopping at the store that hit: the same stores a `set` after the read would write. On a miss everything is still written; nothing changes when no read is pending. The bookkeeping is one small object per `get`, and a promise only when a `set` overlaps it. **`getParams` resolving to a plain object or nothing is a cache miss, not a build failure.** The parameters are now built with `new URLSearchParams(extra ?? undefined)` inside the guarded part, so a plain object such as `{token: 'abc'}` is used as parameters and `undefined`/`null` adds none, and anything else that fails becomes a `NetworkError`. Errors from the request itself are still not wrapped. **Docs:** `Configuration.md` said racing transforms run at most `maxWorkers` at a time; they share `2 * maxWorkers` slots with cache-miss transforms. `Caching.md` describes the write order when a write overlaps a read. Changelog: [Fix] metro-cache: a result written while a read of the same key is still under way no longer writes to the store that read then hits, and an `HttpStore` `getParams` that resolves to a plain object or nothing no longer fails the build Differential Revision: D133210770
1 parent c7b487e commit 6366581

7 files changed

Lines changed: 238 additions & 67 deletions

File tree

‎docs/Caching.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ A typical setup involves:
1717
2. Running [`metro build`](./CLI.md#build-entry) periodically (e.g. in a CI job) to populate the cache, using `HttpStore` (or a custom read/write cache store) in your Metro config.
1818
3. Configuring Metro on development machines to read from the cache, using `HttpGetStore` (or a custom read-only cache store).
1919

20-
Caches are configured with [`cacheStores`](./Configuration.md#cachestores). Metro reads the stores in order and uses the first hit, then writes the result to every store listed before the one that hit (or to all of them, on a miss). So the local cache (e.g. `FileStore`) should usually come first, followed by the remote cache (e.g. `HttpStore` or `HttpGetStore`): a result found only remotely is then copied into the local cache for the next build. For example, on development machines:
20+
Caches are configured with [`cacheStores`](./Configuration.md#cachestores). Metro reads the stores in order and uses the first hit, then writes the result to every store listed before the one that hit (or to all of them, on a miss). If a result is written while a read of the same key is still under way, the stores that read has already missed are written at once, and the rest once the read finishes, again stopping at the one that hit. So the local cache (e.g. `FileStore`) should usually come first, followed by the remote cache (e.g. `HttpStore` or `HttpGetStore`): a result found only remotely is then copied into the local cache for the next build. For example, on development machines:
2121

2222
```js
2323
// metro.config.js

‎docs/Configuration.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -108,7 +108,7 @@ type JsonSerializable = /* Any JSON-serializable value */;
108108

109109
Type: `?number`
110110

111-
If a transform cache read has not returned after this many milliseconds, Metro also queues the transform locally and uses whichever result arrives first. Use it with a remote cache store, so that a slow or congested link can't make builds slower than transforming locally. Local transforms that race a cache read run newest first, at most [`maxWorkers`](#maxworkers) at a time, and are dropped if the cache answers before they start. Defaults to `null` (wait for the cache).
111+
If a transform cache read has not returned after this many milliseconds, Metro also queues the transform locally and uses whichever result arrives first. Use it with a remote cache store, so that a slow or congested link can't make builds slower than transforming locally. Local transforms that race a cache read run newest first, sharing twice as many slots as [`maxWorkers`](#maxworkers) with the transforms of cache misses (which go first), and are dropped if the cache answers before they start. When a local transform wins, Metro writes its result to a later cache store only once that store's pending read has missed. Defaults to `null` (wait for the cache).
112112

113113
#### `cacheVersion`
114114

‎packages/metro-cache/src/Cache.js‎

Lines changed: 118 additions & 59 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,13 @@ import type {CacheStore} from './types';
1313

1414
import {Logger} from 'metro-core';
1515

16+
// A `get` in progress: the store it is reading, and what to call once it has
17+
// finished.
18+
type Reading = {
19+
index: number,
20+
onSettled: ?Array<() => void>,
21+
};
22+
1623
/**
1724
* Main cache class. Receives an array of cache instances, and sequentially
1825
* traverses them to return a previously stored value. It also ensures setting
@@ -23,6 +30,7 @@ import {Logger} from 'metro-core';
2330
export default class Cache<T> {
2431
readonly #stores: ReadonlyArray<CacheStore<T>>;
2532
readonly #hits: WeakMap<Buffer, CacheStore<T>> = new WeakMap();
33+
readonly #reading: WeakMap<Buffer, Reading> = new WeakMap();
2634

2735
constructor(stores: ReadonlyArray<CacheStore<T>>) {
2836
this.#stores = stores;
@@ -31,69 +39,125 @@ export default class Cache<T> {
3139
async get(key: Buffer): Promise<?T> {
3240
const stores = this.#stores;
3341
const length = stores.length;
34-
35-
for (let i = 0; i < length; i++) {
36-
const store = stores[i];
37-
const storeName = store.name ?? store.constructor.name;
38-
const name = storeName + '::' + key.toString('hex');
39-
let value = null;
40-
41-
const logStart = Logger.log(
42-
Logger.createActionStartEntry({
43-
action_name: 'Cache get',
44-
log_entry_label: name,
45-
}),
46-
);
47-
48-
try {
49-
const valueOrPromise = store.get(key);
50-
51-
if (valueOrPromise && typeof valueOrPromise.then === 'function') {
52-
value = await valueOrPromise;
53-
} else {
54-
value = valueOrPromise;
55-
}
56-
} finally {
57-
const hitOrMiss = value != null ? 'hit' : 'miss';
58-
Logger.log({
59-
...Logger.createActionEndEntry(logStart),
60-
action_result: hitOrMiss,
61-
});
62-
63-
// Deprecated - will be removed () - use 'Cache get' and action_result
64-
// (above) instead.
65-
// TODO: T196506422
66-
Logger.log(
67-
Logger.createEntry({
68-
action_name: 'Cache ' + hitOrMiss,
42+
const reading: Reading = {index: 0, onSettled: null};
43+
this.#reading.set(key, reading);
44+
45+
try {
46+
for (let i = 0; i < length; i++) {
47+
reading.index = i;
48+
const store = stores[i];
49+
const storeName = store.name ?? store.constructor.name;
50+
const name = storeName + '::' + key.toString('hex');
51+
let value = null;
52+
53+
const logStart = Logger.log(
54+
Logger.createActionStartEntry({
55+
action_name: 'Cache get',
6956
log_entry_label: name,
7057
}),
7158
);
7259

73-
if (value != null) {
74-
this.#hits.set(key, store);
60+
try {
61+
const valueOrPromise = store.get(key);
7562

76-
return value;
63+
if (valueOrPromise && typeof valueOrPromise.then === 'function') {
64+
value = await valueOrPromise;
65+
} else {
66+
value = valueOrPromise;
67+
}
68+
} finally {
69+
const hitOrMiss = value != null ? 'hit' : 'miss';
70+
Logger.log({
71+
...Logger.createActionEndEntry(logStart),
72+
action_result: hitOrMiss,
73+
});
74+
75+
// Deprecated - will be removed () - use 'Cache get' and action_result
76+
// (above) instead.
77+
// TODO: T196506422
78+
Logger.log(
79+
Logger.createEntry({
80+
action_name: 'Cache ' + hitOrMiss,
81+
log_entry_label: name,
82+
}),
83+
);
84+
85+
if (value != null) {
86+
this.#hits.set(key, store);
87+
88+
return value;
89+
}
7790
}
7891
}
79-
}
8092

81-
return null;
93+
return null;
94+
} finally {
95+
if (this.#reading.get(key) === reading) {
96+
this.#reading.delete(key);
97+
}
98+
for (const onSettled of reading.onSettled ?? []) {
99+
onSettled();
100+
}
101+
}
82102
}
83103

84104
async set(key: Buffer, value: T): Promise<void> {
85-
const stores = this.#stores;
86-
const stop = this.#hits.get(key);
87-
const length = stores.length;
88-
if (length === 0 || stores[0] === stop) {
89-
return;
105+
const reading = this.#reading.get(key);
106+
let failures;
107+
if (reading == null) {
108+
failures = await this.#write(key, value, 0, this.#stopIndex(key));
109+
} else {
110+
// A read of this key is still under way, as when a caller raced it
111+
// against computing the value. The stores before the one it is reading
112+
// have missed and are written now. Whether that store or a later one
113+
// has the value is not known yet, so they are written once the read
114+
// settles, up to the store it hit: the same stores a set after the read
115+
// would write.
116+
const passed = reading.index;
117+
const settled = new Promise<void>(resolve => {
118+
if (reading.onSettled == null) {
119+
reading.onSettled = [];
120+
}
121+
reading.onSettled.push(resolve);
122+
});
123+
const [before, after] = await Promise.all([
124+
this.#write(key, value, 0, passed),
125+
settled.then(() =>
126+
this.#write(key, value, passed, this.#stopIndex(key)),
127+
),
128+
]);
129+
failures = [...before, ...after];
130+
}
131+
132+
if (failures.length > 0) {
133+
throw new AggregateError(
134+
failures.map(failure => failure.error),
135+
`Cache write failed for store(s): ${Array.from(
136+
new Set(failures.map(failure => failure.storeName)),
137+
).join(', ')}`,
138+
);
90139
}
140+
}
141+
142+
// Index of the store that served the last hit for this key: it and the
143+
// stores after it already have the value.
144+
#stopIndex(key: Buffer): number {
145+
const stop = this.#hits.get(key);
146+
const index = stop == null ? -1 : this.#stores.indexOf(stop);
147+
return index === -1 ? this.#stores.length : index;
148+
}
91149

150+
async #write(
151+
key: Buffer,
152+
value: T,
153+
from: number,
154+
to: number,
155+
): Promise<Array<{error: Error, storeName: string}>> {
156+
const stores = this.#stores;
92157
const promises = [];
93-
const writeErrors = [];
94-
const storesWithErrors = new Set<string>();
158+
const failures: Array<{error: Error, storeName: string}> = [];
95159

96-
for (let i = 0; i < length && stores[i] !== stop; i++) {
160+
for (let i = from; i < to; i++) {
97161
const store = stores[i];
98162
const storeName = store.name ?? store.constructor.name;
99163
const name = storeName + '::' + key.toString('hex');
@@ -108,26 +172,21 @@ export default class Cache<T> {
108172
promises.push(
109173
(async () => {
110174
try {
111-
await stores[i].set(key, value);
175+
await store.set(key, value);
112176
Logger.log(Logger.createActionEndEntry(logStart));
113177
} catch (e) {
114178
Logger.log(Logger.createActionEndEntry(logStart, e));
115-
storesWithErrors.add(storeName);
116-
writeErrors.push(
117-
new Error(`Cache write failed for ${name}`, {cause: e}),
118-
);
179+
failures.push({
180+
error: new Error(`Cache write failed for ${name}`, {cause: e}),
181+
storeName,
182+
});
119183
}
120184
})(),
121185
);
122186
}
123187

124188
await Promise.allSettled(promises);
125-
if (writeErrors.length > 0) {
126-
throw new AggregateError(
127-
writeErrors,
128-
`Cache write failed for store(s): ${Array.from(storesWithErrors).join(', ')}`,
129-
);
130-
}
189+
return failures;
131190
}
132191

133192
// Returns true if the current configuration disables the cache, such that

‎packages/metro-cache/src/__tests__/Cache-test.js‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -255,4 +255,62 @@ describe('Cache', () => {
255255
expect(log).toEqual([]);
256256
});
257257
});
258+
describe('set while a get of the same key is under way', () => {
259+
function deferred<T>(): {promise: Promise<T>, resolve: (value: T) => void} {
260+
let resolve: (value: T) => void = () => {};
261+
const promise = new Promise<T>(r => {
262+
resolve = r;
263+
});
264+
return {promise, resolve};
265+
}
266+
267+
async function flush() {
268+
for (let i = 0; i < 10; i++) {
269+
await Promise.resolve();
270+
}
271+
}
272+
273+
test('does not write to the store that then hits', async () => {
274+
const local = createStore();
275+
const remote = createStore();
276+
const remoteRead = deferred<?string>();
277+
remote.get.mockImplementation(() => remoteRead.promise);
278+
const cache = new Cache([local, remote]);
279+
const key = Buffer.from('foo');
280+
281+
const read = cache.get(key);
282+
await flush();
283+
const write = cache.set(key, 'computed');
284+
await flush();
285+
286+
// The store the read has missed is written at once.
287+
expect(local.set).toHaveBeenCalledWith(key, 'computed');
288+
expect(remote.set).not.toHaveBeenCalled();
289+
290+
remoteRead.resolve('cached');
291+
expect(await read).toBe('cached');
292+
await write;
293+
expect(remote.set).not.toHaveBeenCalled();
294+
});
295+
296+
test('writes the rest once the read misses', async () => {
297+
const local = createStore();
298+
const remote = createStore();
299+
const remoteRead = deferred<?string>();
300+
remote.get.mockImplementation(() => remoteRead.promise);
301+
const cache = new Cache([local, remote]);
302+
const key = Buffer.from('foo');
303+
304+
const read = cache.get(key);
305+
await flush();
306+
const write = cache.set(key, 'computed');
307+
await flush();
308+
expect(remote.set).not.toHaveBeenCalled();
309+
310+
remoteRead.resolve(null);
311+
expect(await read).toBe(null);
312+
await write;
313+
expect(remote.set).toHaveBeenCalledWith(key, 'computed');
314+
});
315+
});
258316
});

‎packages/metro-cache/src/stores/HttpStore.js‎

Lines changed: 13 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -431,20 +431,27 @@ export default class HttpStore<T> {
431431
if (getParams == null) {
432432
return request(endpoint.params);
433433
}
434-
// `getParams` may throw rather than reject, or return a plain value, as
435-
// configs are often untyped JS: wrapping it here turns all of these into a
436-
// `NetworkError`, which callers treat like any other failed request.
434+
// Configs are often untyped JS, so `getParams` may throw rather than
435+
// reject, or resolve to a plain object or nothing. Building the parameters
436+
// inside the guarded part turns any of these that fails into a
437+
// `NetworkError`, which callers treat like any other failed request;
438+
// errors from the request itself are left as they are.
437439
return new Promise<URLSearchParams>(resolve => resolve(getParams()))
440+
.then(
441+
extraParams =>
442+
new URLSearchParams([
443+
...endpoint.params,
444+
...new URLSearchParams(extraParams ?? undefined),
445+
]),
446+
)
438447
.catch((error: unknown) => {
439448
throw new NetworkError(
440449
'Could not get request parameters: ' +
441450
(error instanceof Error ? error.message : String(error)),
442451
'EGETPARAMS',
443452
);
444453
})
445-
.then(extraParams =>
446-
request(new URLSearchParams([...endpoint.params, ...extraParams])),
447-
);
454+
.then(request);
448455
}
449456

450457
#withRetries<R>(fn: () => Promise<R>, endpoint: Endpoint): Promise<R> {

‎packages/metro-cache/src/stores/__tests__/HttpGetStore-test.js‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,31 @@ describe('HttpGetStore', () => {
117117
expect(await first).toEqual({foo: 42});
118118
});
119119

120+
// Untyped configs can resolve `getParams` to these; JSON.parse stands in
121+
// for such a config.
122+
test.each([
123+
['a plain object', '{"token": "abc"}', '?token=abc'],
124+
['nothing', 'null', ''],
125+
])(
126+
'accepts a getParams that resolves to %s',
127+
async (_description, json, query) => {
128+
const store = new HttpGetStore({
129+
endpoint: 'http://www.example.com/endpoint',
130+
getParams: () => Promise.resolve(JSON.parse(json)),
131+
});
132+
const promise = store.get(Buffer.from('key'));
133+
for (let i = 0; i < 5; i++) {
134+
await Promise.resolve();
135+
}
136+
const [opts, callback] =
137+
jest.requireMock('node:http').request.mock.calls[0];
138+
callback(responseHttpOk(JSON.stringify({foo: 42})));
139+
140+
expect(await promise).toEqual({foo: 42});
141+
expect(opts.path).toBe('/endpoint/6b6579' + query);
142+
},
143+
);
144+
120145
test("doesn't throw any error and doesn't warn on http status 404 errors", async () => {
121146
const store = new HttpGetStore({endpoint: 'http://example.com'});
122147
const promise = store.get(Buffer.from('key'));

0 commit comments

Comments
 (0)