Skip to content

Commit 5cf8da2

Browse files
GijsWeteringsfacebook-github-bot
authored andcommitted
Remote cache: raise read concurrency and stop per-request CAT minting on VPNless (#2246)
Summary: Remote cache hits are slow mostly because of client-side queueing. On a warm cache Metro starts one `HttpGetStore` GET per module at once (thousands in flight), but `HttpStore` caps its agent at 64 sockets. Most of each logged `Cache get` duration is therefore time spent waiting for a socket; Manifold answers in about 100–150 ms. On VPNless laptops there is a second cap. Every read carries `x-x2pagentd-inject-cat`, and x2pagentd mints a fresh CAT for every such request (`CatInjectFilter`). That holds reads to about 250/s per laptop, whatever the socket count. Changes: - `HttpStore` (metro-cache), both new options documented in `Caching.md`: - `maxSockets` option; the default stays 64. - `getParams` option, called before each request (and each retry); the parameters it resolves to are added to `params`, for short-lived credentials. A throw or rejection fails the request with a `NetworkError`, which `HttpGetStore` treats as a miss. Without `getParams`, requests still start synchronously. - `fb-config/vpnless`: `mintX2PAgentdCat(verifier, validitySeconds)` mints a CAT through x2pagentd's `cats.x2pagentd.local/create` endpoint (the one other tools already use). - `getManifoldHttpConfig` (reads only): - 256 sockets, except on VPNless, which keeps 64: those reads go through the laptop's x2pagentd, which other tools share and which stalls upstream under heavy load. - On VPNless, no inject header. Instead `getParams` mints one `manifold.blobstore` CAT, valid for an hour, reuses it for 50 minutes, then mints again. If minting a replacement fails, reads keep the previous CAT while it is still valid. Reads therefore send the same `crypto_auth_tokens` query parameter that x2pagentd injects today, minted once instead of per read. This does not depend on Manifold accepting reads without a CAT. - Writes are unchanged. Rollout notes: - The `HttpStore` options take effect with the next Metro (MSDK) bump; the prebuilt Metro ignores them until then. The config side loads from the checkout immediately, so until the bump VPNless reads go out with no CAT at all. Manifold accepts that today (verified: reads without a CAT, and with a garbage CAT, return 200 with the payload). - Transform cache keys hash every file under `xplat/js/config/metro` and `xplat/js/config/vpnless`. As with any config change, commits with this change get remote cache misses until the cache warmer has filled them. Changelog: [Feature] metro-cache: `HttpStore` accepts `maxSockets`, to raise or lower its connection limit, and `getParams`, to add query parameters (such as a short-lived credential) to every request Reviewed By: vzaidman Differential Revision: D132331611
1 parent bc93c2a commit 5cf8da2

4 files changed

Lines changed: 137 additions & 8 deletions

File tree

‎docs/Caching.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,10 +44,12 @@ Metro provides these stores for use with [`cacheStores`](./Configuration.md#cach
4444
- **`HttpStore(options)`** is a bare-bones remote cache client that reads (`GET`) and writes (`PUT`) compressed cache artifacts over HTTP or HTTPS.
4545
- **`options.endpoint: string`**: Base URL of the cache server. For example, with `'http://www.example.com/endpoint'`, requests go to URLs such as `http://www.example.com/endpoint/c083bff944879d9f528cf185eba0f4962fd4e5b1a6a7d2a1c0bd42c3c0d5f0e1b6a9e7c8` (the hex-encoded cache key).
4646
- **`options.timeout: number`**: Request timeout in milliseconds. Defaults to 5000.
47+
- **`options.maxSockets: number`**: Maximum concurrent connections to the cache server. Requests beyond this wait for a free connection, and that wait counts towards the request's duration. Defaults to 64.
4748
- **`options.family: 4 | 6`**: Same as the `family` parameter to Node's [`http.request`](https://nodejs.org/api/http.html#httprequesturl-options-callback).
4849
- **`options.cert`, `options.ca`, `options.key`**: HTTPS options passed directly to [Node's built-in HTTPS client](https://nodejs.org/api/https.html).
4950
- **`options.headers: {[string]: string}`**: HTTP headers sent with every request.
5051
- **`options.params: URLSearchParams`**: Query parameters appended to every request URL.
52+
- **`options.getParams: () => Promise<URLSearchParams>`**: Called before every request (and retry); the parameters it resolves to are added to `options.params`. Use it for short-lived credentials such as auth tokens that need refreshing. If it throws or rejects, the request fails with a network error.
5153
- **`options.additionalSuccessStatuses: Array<number>`**: Additional HTTP status codes treated as success.
5254
- **`options.maxAttempts: number`**: Maximum attempts per request, with exponential backoff. Defaults to 1 (no retries).
5355
- **`options.retryStatuses: Set<number>`**: HTTP status codes that trigger a retry.

‎packages/metro-cache/src/stores/HttpStore.js‎

Lines changed: 50 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -27,10 +27,16 @@ type EndpointOptions = {
2727
endpoint: string,
2828
family?: 4 | 6,
2929
timeout?: number,
30+
maxSockets?: number,
3031
key?: string | ReadonlyArray<string> | Buffer | ReadonlyArray<Buffer>,
3132
cert?: string | ReadonlyArray<string> | Buffer | ReadonlyArray<Buffer>,
3233
ca?: string | ReadonlyArray<string> | Buffer | ReadonlyArray<Buffer>,
3334
params?: URLSearchParams,
35+
/**
36+
* Called before every request; the parameters it resolves to are added to
37+
* `params`. Use it for short-lived credentials.
38+
*/
39+
getParams?: () => Promise<URLSearchParams>,
3440
headers?: {[key: string]: string},
3541
additionalSuccessStatuses?: ReadonlyArray<number>,
3642
/**
@@ -55,6 +61,7 @@ type Endpoint = {
5561
port: number,
5662
agent: http$Agent<tls$TLSSocket> | http$Agent<net$Socket>,
5763
params: URLSearchParams,
64+
getParams: ?() => Promise<URLSearchParams>,
5865
headers?: {[key: string]: string},
5966
timeout: number,
6067
additionalSuccessStatuses: ReadonlySet<number>,
@@ -96,8 +103,8 @@ export default class HttpStore<T> {
96103
family: options.family,
97104
keepAlive: true,
98105
keepAliveMsecs: options.timeout || 5000,
99-
maxFreeSockets: 64,
100-
maxSockets: 64,
106+
maxFreeSockets: options.maxSockets ?? 64,
107+
maxSockets: options.maxSockets ?? 64,
101108
};
102109

103110
if (options.key != null) {
@@ -138,6 +145,7 @@ export default class HttpStore<T> {
138145
),
139146
agent,
140147
debug: options.debug ?? false,
148+
getParams: options.getParams,
141149
headers: options.headers,
142150
host: uri.hostname,
143151
maxAttempts: options.maxAttempts ?? 1,
@@ -152,12 +160,18 @@ export default class HttpStore<T> {
152160
}
153161

154162
get(key: Buffer): Promise<?T> {
155-
return this.#withRetries(() => this.#getOnce(key), this.#getEndpoint);
163+
return this.#withRetries(
164+
() =>
165+
this.#withParams(this.#getEndpoint, params =>
166+
this.#getOnce(key, params),
167+
),
168+
this.#getEndpoint,
169+
);
156170
}
157171

158-
#getOnce(key: Buffer): Promise<?T> {
172+
#getOnce(key: Buffer, params: URLSearchParams): Promise<?T> {
159173
return new Promise((resolve, reject) => {
160-
let searchParamsString = this.#getEndpoint.params.toString();
174+
let searchParamsString = params.toString();
161175
if (searchParamsString != '') {
162176
searchParamsString = '?' + searchParamsString;
163177
}
@@ -275,16 +289,19 @@ export default class HttpStore<T> {
275289

276290
set(key: Buffer, value: T): Promise<void> {
277291
return this.#withRetries(
278-
() => this.#setOnce(key, value),
292+
() =>
293+
this.#withParams(this.#setEndpoint, params =>
294+
this.#setOnce(key, value, params),
295+
),
279296
this.#setEndpoint,
280297
);
281298
}
282299

283-
#setOnce(key: Buffer, value: T): Promise<void> {
300+
#setOnce(key: Buffer, value: T, params: URLSearchParams): Promise<void> {
284301
return new Promise((resolve, reject) => {
285302
const gzip = zlib.createGzip(ZLIB_OPTIONS);
286303

287-
let searchParamsString = this.#setEndpoint.params.toString();
304+
let searchParamsString = params.toString();
288305
if (searchParamsString != '') {
289306
searchParamsString = '?' + searchParamsString;
290307
}
@@ -400,6 +417,31 @@ export default class HttpStore<T> {
400417
// Not implemented.
401418
}
402419

420+
// Requests go out synchronously unless `getParams` is set.
421+
#withParams<R>(
422+
endpoint: Endpoint,
423+
request: (params: URLSearchParams) => Promise<R>,
424+
): Promise<R> {
425+
const getParams = endpoint.getParams;
426+
if (getParams == null) {
427+
return request(endpoint.params);
428+
}
429+
// `getParams` may throw rather than reject, or return a plain value, as
430+
// configs are often untyped JS: wrapping it here turns all of these into a
431+
// `NetworkError`, which callers treat like any other failed request.
432+
return new Promise<URLSearchParams>(resolve => resolve(getParams()))
433+
.catch((error: unknown) => {
434+
throw new NetworkError(
435+
'Could not get request parameters: ' +
436+
(error instanceof Error ? error.message : String(error)),
437+
'EGETPARAMS',
438+
);
439+
})
440+
.then(extraParams =>
441+
request(new URLSearchParams([...endpoint.params, ...extraParams])),
442+
);
443+
}
444+
403445
#withRetries<R>(fn: () => Promise<R>, endpoint: Endpoint): Promise<R> {
404446
if (endpoint.maxAttempts === 1) {
405447
return fn();

‎packages/metro-cache/src/stores/__tests__/HttpGetStore-test.js‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,18 @@ describe('HttpGetStore', () => {
6969
expect(warningMessages.length).toBe(0);
7070
});
7171

72+
test('treats a getParams that throws as a miss', async () => {
73+
const store = new HttpGetStore({
74+
endpoint: 'http://www.example.com/endpoint',
75+
getParams: () => {
76+
throw new Error('token file missing');
77+
},
78+
});
79+
80+
expect(await store.get(Buffer.from('key'))).toBe(null);
81+
expect(jest.requireMock('node:http').request).not.toHaveBeenCalled();
82+
});
83+
7284
test("doesn't throw any error and doesn't warn on http status 404 errors", async () => {
7385
const store = new HttpGetStore({endpoint: 'http://example.com'});
7486
const promise = store.get(Buffer.from('key'));

‎packages/metro-cache/src/stores/__tests__/HttpStore-test.js‎

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,79 @@ describe('HttpStore', () => {
102102
expect(await promise).toEqual({foo: 42});
103103
});
104104

105+
test('limits the agent to maxSockets, defaulting to 64', () => {
106+
const {Agent} = jest.requireMock('node:http');
107+
expect(new HttpStore({endpoint: 'http://example.com'})).toBeInstanceOf(
108+
HttpStore,
109+
);
110+
expect(
111+
new HttpStore({endpoint: 'http://example.com', maxSockets: 256}),
112+
).toBeInstanceOf(HttpStore);
113+
114+
expect(Agent.mock.calls[0][0]).toMatchObject({
115+
maxSockets: 64,
116+
maxFreeSockets: 64,
117+
});
118+
expect(Agent.mock.calls[2][0]).toMatchObject({
119+
maxSockets: 256,
120+
maxFreeSockets: 256,
121+
});
122+
});
123+
124+
test('adds the params from getParams to every request', async () => {
125+
const getParams = jest.fn(() =>
126+
Promise.resolve(new URLSearchParams({token: 'abc'})),
127+
);
128+
const store = new HttpStore({
129+
endpoint: 'http://www.example.com/endpoint',
130+
params: new URLSearchParams({bucket: 'b'}),
131+
getParams,
132+
});
133+
const promise = store.get(Buffer.from('key'));
134+
// The request starts once the params have resolved.
135+
for (let i = 0; i < 5; i++) {
136+
await Promise.resolve();
137+
}
138+
const [opts, callback] =
139+
jest.requireMock('node:http').request.mock.calls[0];
140+
141+
expect(getParams).toHaveBeenCalledTimes(1);
142+
expect(opts.path).toEqual('/endpoint/6b6579?bucket=b&token=abc');
143+
144+
callback(responseHttpOk(JSON.stringify({foo: 42})));
145+
jest.runAllTimers();
146+
147+
expect(await promise).toEqual({foo: 42});
148+
});
149+
150+
test('rejects with a NetworkError when getParams rejects', async () => {
151+
const store = new HttpStore({
152+
endpoint: 'http://example.com',
153+
getParams: () => Promise.reject(new Error('mint failed')),
154+
});
155+
156+
const error = await store.get(Buffer.from('key')).catch(e => e);
157+
158+
expect(error).toBeInstanceOf(HttpStore.NetworkError);
159+
expect(error.message).toMatch(/mint failed/);
160+
expect(jest.requireMock('node:http').request).not.toHaveBeenCalled();
161+
});
162+
163+
test('rejects with a NetworkError when getParams throws', async () => {
164+
const store = new HttpStore({
165+
endpoint: 'http://example.com',
166+
getParams: () => {
167+
throw new Error('token file missing');
168+
},
169+
});
170+
171+
const error = await store.get(Buffer.from('key')).catch(e => e);
172+
173+
expect(error).toBeInstanceOf(HttpStore.NetworkError);
174+
expect(error.message).toMatch(/token file missing/);
175+
expect(jest.requireMock('node:http').request).not.toHaveBeenCalled();
176+
});
177+
105178
test('rejects when an HTTP different from 200 is returned', done => {
106179
const store = new HttpStore({endpoint: 'http://example.com'});
107180
const promise = store.get(Buffer.from('key'));

0 commit comments

Comments
 (0)