Skip to content

Commit 2cdff9f

Browse files
GijsWeteringsfacebook-github-bot
authored andcommitted
Remote cache: raise read concurrency and stop per-request CAT minting on VPNless
Summary: Remote cache hits are slow mostly because of client-side queueing. On a warm cache Metro starts one `HttpGetStore` GET per module at once (thousands in flight), but `HttpStore` caps its agent at 64 sockets. Most of each logged `Cache get` duration is therefore time spent waiting for a socket; Manifold answers in about 100–150 ms. On VPNless laptops there is a second cap. Every read carries `x-x2pagentd-inject-cat`, and x2pagentd mints a fresh CAT for every such request (`CatInjectFilter`). That holds reads to about 250/s per laptop, whatever the socket count. Changes: - `HttpStore` (metro-cache), both new options documented in `Caching.md`: - `maxSockets` option; the default stays 64. - `getParams` option, called before each request (and each retry); the parameters it resolves to are added to `params`, for short-lived credentials. A rejection fails the request with a `NetworkError`, which `HttpGetStore` treats as a miss. Without `getParams`, requests still start synchronously. - `fb-config/vpnless`: `mintX2PAgentdCat(verifier, validitySeconds)` mints a CAT through x2pagentd's `cats.x2pagentd.local/create` endpoint (the one other tools already use). - `getManifoldHttpConfig` (reads only): - 256 sockets. - On VPNless, no inject header. Instead `getParams` mints one `manifold.blobstore` CAT, valid for an hour, reuses it for 50 minutes, then mints again. Reads therefore send the same `crypto_auth_tokens` query parameter that x2pagentd injects today, minted once instead of per read. This does not depend on Manifold accepting reads without a CAT. - Writes are unchanged. Rollout notes: - The `HttpStore` options take effect with the next Metro (MSDK) bump; the prebuilt Metro ignores them until then. The config side loads from the checkout immediately, so until the bump VPNless reads go out with no CAT at all. Manifold accepts that today (verified: reads without a CAT, and with a garbage CAT, return 200 with the payload). - Transform cache keys hash every file under `xplat/js/config/metro` and `xplat/js/config/vpnless`. As with any config change, commits with this change get remote cache misses until the cache warmer has filled them. Changelog: [Feature] metro-cache: `HttpStore` accepts `maxSockets`, to raise or lower its connection limit, and `getParams`, to add query parameters (such as a short-lived credential) to every request Differential Revision: D132331611
1 parent a21bdde commit 2cdff9f

3 files changed

Lines changed: 104 additions & 8 deletions

File tree

‎docs/Caching.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,10 +44,12 @@ Metro provides these stores for use with [`cacheStores`](./Configuration.md#cach
4444
- **`HttpStore(options)`** is a bare-bones remote cache client that reads (`GET`) and writes (`PUT`) compressed cache artifacts over HTTP or HTTPS.
4545
- **`options.endpoint: string`**: Base URL of the cache server. For example, with `'http://www.example.com/endpoint'`, requests go to URLs such as `http://www.example.com/endpoint/c083bff944879d9f528cf185eba0f4962fd4e5b1a6a7d2a1c0bd42c3c0d5f0e1b6a9e7c8` (the hex-encoded cache key).
4646
- **`options.timeout: number`**: Request timeout in milliseconds. Defaults to 5000.
47+
- **`options.maxSockets: number`**: Maximum concurrent connections to the cache server. Requests beyond this wait for a free connection, and that wait counts towards the request's duration. Defaults to 64.
4748
- **`options.family: 4 | 6`**: Same as the `family` parameter to Node's [`http.request`](https://nodejs.org/api/http.html#httprequesturl-options-callback).
4849
- **`options.cert`, `options.ca`, `options.key`**: HTTPS options passed directly to [Node's built-in HTTPS client](https://nodejs.org/api/https.html).
4950
- **`options.headers: {[string]: string}`**: HTTP headers sent with every request.
5051
- **`options.params: URLSearchParams`**: Query parameters appended to every request URL.
52+
- **`options.getParams: () => Promise<URLSearchParams>`**: Called before every request (and retry); the parameters it resolves to are added to `options.params`. Use it for short-lived credentials such as auth tokens that need refreshing. If it rejects, the request fails with a network error.
5153
- **`options.additionalSuccessStatuses: Array<number>`**: Additional HTTP status codes treated as success.
5254
- **`options.maxAttempts: number`**: Maximum attempts per request, with exponential backoff. Defaults to 1 (no retries).
5355
- **`options.retryStatuses: Set<number>`**: HTTP status codes that trigger a retry.

‎packages/metro-cache/src/stores/HttpStore.js‎

Lines changed: 47 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -27,10 +27,16 @@ type EndpointOptions = {
2727
endpoint: string,
2828
family?: 4 | 6,
2929
timeout?: number,
30+
maxSockets?: number,
3031
key?: string | ReadonlyArray<string> | Buffer | ReadonlyArray<Buffer>,
3132
cert?: string | ReadonlyArray<string> | Buffer | ReadonlyArray<Buffer>,
3233
ca?: string | ReadonlyArray<string> | Buffer | ReadonlyArray<Buffer>,
3334
params?: URLSearchParams,
35+
/**
36+
* Called before every request; the parameters it resolves to are added to
37+
* `params`. Use it for short-lived credentials.
38+
*/
39+
getParams?: () => Promise<URLSearchParams>,
3440
headers?: {[key: string]: string},
3541
additionalSuccessStatuses?: ReadonlyArray<number>,
3642
/**
@@ -55,6 +61,7 @@ type Endpoint = {
5561
port: number,
5662
agent: http$Agent<tls$TLSSocket> | http$Agent<net$Socket>,
5763
params: URLSearchParams,
64+
getParams: ?() => Promise<URLSearchParams>,
5865
headers?: {[key: string]: string},
5966
timeout: number,
6067
additionalSuccessStatuses: ReadonlySet<number>,
@@ -96,8 +103,8 @@ export default class HttpStore<T> {
96103
family: options.family,
97104
keepAlive: true,
98105
keepAliveMsecs: options.timeout || 5000,
99-
maxFreeSockets: 64,
100-
maxSockets: 64,
106+
maxFreeSockets: options.maxSockets ?? 64,
107+
maxSockets: options.maxSockets ?? 64,
101108
};
102109

103110
if (options.key != null) {
@@ -138,6 +145,7 @@ export default class HttpStore<T> {
138145
),
139146
agent,
140147
debug: options.debug ?? false,
148+
getParams: options.getParams,
141149
headers: options.headers,
142150
host: uri.hostname,
143151
maxAttempts: options.maxAttempts ?? 1,
@@ -152,12 +160,18 @@ export default class HttpStore<T> {
152160
}
153161

154162
get(key: Buffer): Promise<?T> {
155-
return this.#withRetries(() => this.#getOnce(key), this.#getEndpoint);
163+
return this.#withRetries(
164+
() =>
165+
this.#withParams(this.#getEndpoint, params =>
166+
this.#getOnce(key, params),
167+
),
168+
this.#getEndpoint,
169+
);
156170
}
157171

158-
#getOnce(key: Buffer): Promise<?T> {
172+
#getOnce(key: Buffer, params: URLSearchParams): Promise<?T> {
159173
return new Promise((resolve, reject) => {
160-
let searchParamsString = this.#getEndpoint.params.toString();
174+
let searchParamsString = params.toString();
161175
if (searchParamsString != '') {
162176
searchParamsString = '?' + searchParamsString;
163177
}
@@ -275,16 +289,19 @@ export default class HttpStore<T> {
275289

276290
set(key: Buffer, value: T): Promise<void> {
277291
return this.#withRetries(
278-
() => this.#setOnce(key, value),
292+
() =>
293+
this.#withParams(this.#setEndpoint, params =>
294+
this.#setOnce(key, value, params),
295+
),
279296
this.#setEndpoint,
280297
);
281298
}
282299

283-
#setOnce(key: Buffer, value: T): Promise<void> {
300+
#setOnce(key: Buffer, value: T, params: URLSearchParams): Promise<void> {
284301
return new Promise((resolve, reject) => {
285302
const gzip = zlib.createGzip(ZLIB_OPTIONS);
286303

287-
let searchParamsString = this.#setEndpoint.params.toString();
304+
let searchParamsString = params.toString();
288305
if (searchParamsString != '') {
289306
searchParamsString = '?' + searchParamsString;
290307
}
@@ -400,6 +417,28 @@ export default class HttpStore<T> {
400417
// Not implemented.
401418
}
402419

420+
// Requests go out synchronously unless `getParams` is set.
421+
#withParams<R>(
422+
endpoint: Endpoint,
423+
request: (params: URLSearchParams) => Promise<R>,
424+
): Promise<R> {
425+
const getParams = endpoint.getParams;
426+
if (getParams == null) {
427+
return request(endpoint.params);
428+
}
429+
return getParams().then(
430+
extraParams =>
431+
request(new URLSearchParams([...endpoint.params, ...extraParams])),
432+
(error: unknown) => {
433+
throw new NetworkError(
434+
'Could not get request parameters: ' +
435+
(error instanceof Error ? error.message : String(error)),
436+
'EGETPARAMS',
437+
);
438+
},
439+
);
440+
}
441+
403442
#withRetries<R>(fn: () => Promise<R>, endpoint: Endpoint): Promise<R> {
404443
if (endpoint.maxAttempts === 1) {
405444
return fn();

‎packages/metro-cache/src/stores/__tests__/HttpStore-test.js‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,61 @@ describe('HttpStore', () => {
102102
expect(await promise).toEqual({foo: 42});
103103
});
104104

105+
test('limits the agent to maxSockets, defaulting to 64', () => {
106+
const {Agent} = jest.requireMock('node:http');
107+
expect(new HttpStore({endpoint: 'http://example.com'})).toBeInstanceOf(
108+
HttpStore,
109+
);
110+
expect(
111+
new HttpStore({endpoint: 'http://example.com', maxSockets: 256}),
112+
).toBeInstanceOf(HttpStore);
113+
114+
expect(Agent.mock.calls[0][0]).toMatchObject({
115+
maxSockets: 64,
116+
maxFreeSockets: 64,
117+
});
118+
expect(Agent.mock.calls[2][0]).toMatchObject({
119+
maxSockets: 256,
120+
maxFreeSockets: 256,
121+
});
122+
});
123+
124+
test('adds the params from getParams to every request', async () => {
125+
const getParams = jest.fn(() =>
126+
Promise.resolve(new URLSearchParams({token: 'abc'})),
127+
);
128+
const store = new HttpStore({
129+
endpoint: 'http://www.example.com/endpoint',
130+
params: new URLSearchParams({bucket: 'b'}),
131+
getParams,
132+
});
133+
const promise = store.get(Buffer.from('key'));
134+
await Promise.resolve();
135+
const [opts, callback] =
136+
jest.requireMock('node:http').request.mock.calls[0];
137+
138+
expect(getParams).toHaveBeenCalledTimes(1);
139+
expect(opts.path).toEqual('/endpoint/6b6579?bucket=b&token=abc');
140+
141+
callback(responseHttpOk(JSON.stringify({foo: 42})));
142+
jest.runAllTimers();
143+
144+
expect(await promise).toEqual({foo: 42});
145+
});
146+
147+
test('rejects with a NetworkError when getParams rejects', async () => {
148+
const store = new HttpStore({
149+
endpoint: 'http://example.com',
150+
getParams: () => Promise.reject(new Error('mint failed')),
151+
});
152+
153+
const error = await store.get(Buffer.from('key')).catch(e => e);
154+
155+
expect(error).toBeInstanceOf(HttpStore.NetworkError);
156+
expect(error.message).toMatch(/mint failed/);
157+
expect(jest.requireMock('node:http').request).not.toHaveBeenCalled();
158+
});
159+
105160
test('rejects when an HTTP different from 200 is returned', done => {
106161
const store = new HttpStore({endpoint: 'http://example.com'});
107162
const promise = store.get(Buffer.from('key'));

0 commit comments

Comments
 (0)