Skip to content

Commit aa75dc8

Browse files
committed
metro-file-map: Let plugins report what changed in a batch, on the change event
Stacked on #1952. ## Summary A file map plugin often knows things about a batch of changes that nothing else can work out. The motivating case is Haste - #1952 records the Haste names a resolution looks up, and to invalidate from those a consumer needs to know which names were rebound by a change. It can't derive that from the `ChangeEvent`, which carries paths and modified times, and by the time it sees the event a removed file's name is already gone from the map. `HastePlugin` has it to hand - its `onChanged` is given each removed file's old name and each added file's new one. This gives plugins a way to say so. `FileMapPlugin.onChanged` may now return a summary, and the file map publishes what each plugin returned as `ChangeEvent.pluginChanges`, by plugin name. A plugin that returns nothing is absent, so existing plugins are unaffected - the summary is a third type parameter on `FileMapPlugin` that defaults to `void`. `getPluginChanges(event, plugin)` reads an entry with the plugin's own summary type, so a consumer never writes the name or casts. The summary describes exactly the batch in `changes`. Plugins are already updated synchronously, immediately before the emit, so there's no ordering for a consumer to get wrong - which is the problem with the alternative of asking the plugin afterwards. There are two emit sites: the batched watch path updates plugins inline, and a recrawl updates them inside `#applyFileDelta`, which now returns what they reported along with the changes. The initial build goes through `#applyFileDelta` too but emits nothing, so its summaries go unused. `HastePlugin` is the first user. It reports `changedNames`, the names bound or unbound for any platform by the files added and removed - a bare name, matching what #1952 records. Plugin names now have to be unique across all of a file map's plugins. That was only checked for plugins with a worker, but persisted plugin state is keyed by name for every plugin, so two that shared a name would already have overwritten each other's state. Two things worth knowing: - On a cold start `onChanged` sees every file as added, so `HastePlugin` builds a set of every name for a summary nobody reads. That's nothing where Haste is inert, which is Metro's default, and transient otherwise. If it matters, the alternative is an optional method the file map calls only for a batch it's about to emit. - A modified file reaches plugins in `modifiedFiles` with its new data only, and `HastePlugin.onChanged` doesn't read that list. So an edit that changes a file's Haste name (a docblock, or a global package's `name`) is neither applied to the map nor reported here. That predates this, and can't happen with a path-based `hasteImpl`. `FileSystemChangeAggregator` keeps each file's pre-batch metadata, so giving plugins the old value is feasible, separately. Nothing consumes this yet. ## Test plan ``` yarn jest packages/metro-file-map packages/metro-resolver packages/metro/src/node-haste packages/metro/src/DeltaBundler packages/metro/src/integration_tests packages/metro/src/__tests__ packages/metro/src/Server yarn flow check yarn typecheck-ts yarn verify-api-snapshots ``` - `HastePlugin-test.js`: `onChanged` reports the names bound and unbound, and nothing when no file with a name was added or removed. - `index-test.js`: in watch mode an added and a removed file put both names on the event, read back through `getPluginChanges`, and plugins with nothing to say are absent. A recrawl publishes what plugins reported too. Two plugins sharing a name throw, with or without a worker. Changelog: Internal
1 parent d690032 commit aa75dc8

7 files changed

Lines changed: 229 additions & 42 deletions

File tree

‎packages/metro-file-map/API.md‎

Lines changed: 26 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,7 @@ export type CacheManagerWriteOptions = Readonly<{
4747
export type ChangeEvent = Readonly<{
4848
logger?: null | undefined | RootPerfLogger;
4949
changes: ReadonlyFileSystemChanges<Readonly<ChangedFileMetadata>>;
50+
pluginChanges: ReadonlyMap<string, unknown>;
5051
rootDir: string;
5152
}>;
5253

@@ -184,6 +185,25 @@ interface FileSystem_2 {
184185
}
185186
export {FileSystem_2 as FileSystem};
186187

188+
export function getPluginChanges<ChangeSummary>(
189+
event: ChangeEvent,
190+
plugin: FileMapPlugin<
191+
/**
192+
* > | plugin: FileMapPlugin<empty, empty, ChangeSummary>,
193+
* | ^^^^^ Unsupported feature: Translating "empty type" is currently not supported.
194+
**/
195+
any,
196+
/**
197+
* > | plugin: FileMapPlugin<empty, empty, ChangeSummary>,
198+
* | ^^^^^ Unsupported feature: Translating "empty type" is currently not supported.
199+
**/
200+
any,
201+
ChangeSummary
202+
>,
203+
): ChangeSummary | void;
204+
205+
export type HasteChanges = Readonly<{changedNames: ReadonlySet<string>}>;
206+
187207
export class HasteConflictsError extends Error {
188208
constructor(conflicts: ReadonlyArray<HasteConflict>);
189209
getDetailedMessage(pathsRelativeToRoot: null | undefined | string): string;
@@ -202,7 +222,7 @@ export type HasteMapItem = {
202222
[platform: string]: HasteMapItemMetadata;
203223
};
204224

205-
export class HastePlugin implements HasteMap, FileMapPlugin<null, string | null> {
225+
export class HastePlugin implements HasteMap, FileMapPlugin<null, string | null, HasteChanges | void> {
206226
constructor(options: HasteMapOptions);
207227
assertValid(): void;
208228
computeConflicts(): Array<HasteConflict>;
@@ -214,7 +234,7 @@ export class HastePlugin implements HasteMap, FileMapPlugin<null, string | null>
214234
getWorker(): FileMapPluginWorker;
215235
initialize(opts: FileMapPluginInitOptions<null, string | null>): Promise<void>;
216236
readonly name: 'haste';
217-
onChanged(delta: ReadonlyFileSystemChanges<null | undefined | string>): void;
237+
onChanged(delta: ReadonlyFileSystemChanges<null | undefined | string>): HasteChanges | void;
218238
setModule(id: string, module: HasteMapItemMetadata): void;
219239
}
220240

@@ -240,15 +260,16 @@ export type HealthCheckResult =
240260

241261
export type InputFileMapPlugin = FileMapPlugin<
242262
/**
243-
* > | export type InputFileMapPlugin = FileMapPlugin<empty, empty>;
263+
* > | export type InputFileMapPlugin = FileMapPlugin<empty, empty, unknown>;
244264
* | ^^^^^ Unsupported feature: Translating "empty type" is currently not supported.
245265
**/
246266
any,
247267
/**
248-
* > | export type InputFileMapPlugin = FileMapPlugin<empty, empty>;
268+
* > | export type InputFileMapPlugin = FileMapPlugin<empty, empty, unknown>;
249269
* | ^^^^^ Unsupported feature: Translating "empty type" is currently not supported.
250270
**/
251-
any
271+
any,
272+
unknown
252273
>;
253274

254275
export type InputOptions = Readonly<{

‎packages/metro-file-map/src/__tests__/index-test.js‎

Lines changed: 51 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ import type {
2727
} from '../flow-types';
2828
import type {DependencyPlugin} from '../index';
2929
import type {default as FileMapT} from '../index';
30-
import type {HasteMapOptions} from '../plugins/HastePlugin';
30+
import type HastePluginT, {HasteMapOptions} from '../plugins/HastePlugin';
3131
import type {MockMapOptions} from '../plugins/MockPlugin';
3232
import typeof WorkerModule from '../worker';
3333

@@ -278,7 +278,7 @@ let buildNewFileMap: (
278278
) => Promise<{
279279
...BuildResult,
280280
fileMap: FileMapT,
281-
hasteMap: HasteMap,
281+
hasteMap: HastePluginT,
282282
mockMap: ?MockMap,
283283
dependencyPlugin: ?DependencyPlugin,
284284
}>;
@@ -434,6 +434,24 @@ describe('FileMap', () => {
434434
expect(FileMap.H).toBe(require('../constants').default);
435435
});
436436

437+
test('throws on plugins that share a name, with or without a worker', () => {
438+
const MockPlugin = require('../plugins/MockPlugin').default;
439+
const mockConfig = {
440+
console: globalThis.console,
441+
rootDir: defaultConfig.rootDir,
442+
mocksPattern: /__mocks__/,
443+
throwOnModuleCollision: false,
444+
};
445+
expect(
446+
() =>
447+
new FileMap({
448+
...defaultConfig,
449+
// MockPlugin has no worker, so holds no plugin data slot
450+
plugins: [new MockPlugin(mockConfig), new MockPlugin(mockConfig)],
451+
}),
452+
).toThrow('metro-file-map: Duplicate plugin name: mocks');
453+
});
454+
437455
test('ignores files given a pattern', async () => {
438456
mockFs[path.join('/', 'project', 'fruits', 'Kiwi.js')] = `
439457
// Kiwi!
@@ -1760,6 +1778,28 @@ describe('FileMap', () => {
17601778
}
17611779
});
17621780

1781+
fm_it(
1782+
'publishes what each plugin reports about a batch on the change event',
1783+
async ({fileMap, hasteMap}) => {
1784+
const getPluginChanges = require('../lib/getPluginChanges').default;
1785+
const fruitsRoot = path.join('/', 'project', 'fruits');
1786+
mockFs[path.join(fruitsRoot, 'Tomato.js')] = '// Tomato!';
1787+
mockEmitters[fruitsRoot].emitFileEvent({
1788+
event: 'touch',
1789+
relativePath: 'Tomato.js',
1790+
metadata: MOCK_CHANGE_FILE,
1791+
});
1792+
mockDeleteFile(fruitsRoot, 'Banana.js');
1793+
const changeEvent = await waitForItToChange(fileMap);
1794+
1795+
expect(getPluginChanges(changeEvent, hasteMap)).toEqual({
1796+
changedNames: new Set(['Tomato', 'Banana']),
1797+
});
1798+
// Only plugins with something to say are present
1799+
expect([...changeEvent.pluginChanges.keys()]).toEqual([hasteMap.name]);
1800+
},
1801+
);
1802+
17631803
function mockDeleteFile(root: string, relativePath: string) {
17641804
const e = mockEmitters[root];
17651805
e.emitFileEvent({event: 'delete', relativePath});
@@ -1774,7 +1814,7 @@ describe('FileMap', () => {
17741814

17751815
function fm_it(
17761816
title: string,
1777-
fn: (fm: Readonly<{fileMap: FileMap, hasteMap: HasteMap}>) => unknown,
1817+
fn: (fm: Readonly<{fileMap: FileMap, hasteMap: HastePluginT}>) => unknown,
17781818
options?: FileMapTestOptions = {},
17791819
): void {
17801820
options = options || {};
@@ -1797,7 +1837,7 @@ describe('FileMap', () => {
17971837

17981838
fm_it.only = (
17991839
title: string,
1800-
fn: (fm: Readonly<{fileMap: FileMap, hasteMap: HasteMap}>) => unknown,
1840+
fn: (fm: Readonly<{fileMap: FileMap, hasteMap: HastePluginT}>) => unknown,
18011841
options?: FileMapTestOptions,
18021842
): void => fm_it(title, fn, {...options, only: true});
18031843

@@ -2628,7 +2668,13 @@ describe('FileMap', () => {
26282668
relativePath: 'tropical',
26292669
});
26302670

2631-
await waitForItToChange(fileMap);
2671+
const changeEvent = await waitForItToChange(fileMap);
2672+
2673+
// Plugins are updated inside the recrawl, and what they report
2674+
// still reaches the event
2675+
expect(
2676+
require('../lib/getPluginChanges').default(changeEvent, hasteMap),
2677+
).toEqual({changedNames: new Set(['Mango', 'Papaya'])});
26322678

26332679
// Verify crawl was called with the correct directory
26342680
expect(mockNodeCrawler).toHaveBeenNthCalledWith(

‎packages/metro-file-map/src/flow-types.js‎

Lines changed: 18 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,12 @@ export type ChangedFileMetadata = Readonly<{
9797
export type ChangeEvent = Readonly<{
9898
logger?: ?RootPerfLogger,
9999
changes: ReadonlyFileSystemChanges<Readonly<ChangedFileMetadata>>,
100+
/**
101+
* What each plugin reported about this batch of changes, by plugin name. A
102+
* plugin whose `onChanged` returned nothing is absent. Read an entry with
103+
* `getPluginChanges`, which knows its type.
104+
*/
105+
pluginChanges: ReadonlyMap<string, unknown>,
100106
rootDir: string,
101107
}>;
102108

@@ -264,19 +270,29 @@ export type V8Serializable = V8SerializablePrimitive | V8SerializableCollection;
264270
export interface FileMapPlugin<
265271
in SerializableState extends void | V8Serializable = void | V8Serializable,
266272
in PerFileData extends void | V8Serializable = void | V8Serializable,
273+
out ChangeSummary = void,
267274
> {
275+
// Must be unique among the plugins of a file map.
268276
readonly name: string;
269277
initialize(
270278
initOptions: FileMapPluginInitOptions<SerializableState, PerFileData>,
271279
): Promise<void>;
272280
assertValid(): void;
273-
onChanged(changes: ReadonlyFileSystemChanges<?PerFileData>): void;
281+
/**
282+
* Update the plugin's state for a batch of changes. It is called
283+
* synchronously, immediately before the batch is emitted as a `ChangeEvent`.
284+
*
285+
* A plugin may return a summary of what changed in terms only it knows,
286+
* such as the Haste names that were rebound, for the file map to publish as
287+
* `ChangeEvent.pluginChanges`. Return nothing if there is nothing to say.
288+
*/
289+
onChanged(changes: ReadonlyFileSystemChanges<?PerFileData>): ChangeSummary;
274290
getSerializableSnapshot(): void | V8Serializable;
275291
getCacheKey(): string;
276292
getWorker(): ?FileMapPluginWorker;
277293
}
278294

279-
export type InputFileMapPlugin = FileMapPlugin<empty, empty>;
295+
export type InputFileMapPlugin = FileMapPlugin<empty, empty, unknown>;
280296

281297
export interface MetadataWorker {
282298
processFile(

‎packages/metro-file-map/src/index.js‎

Lines changed: 50 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -122,7 +122,7 @@ type InternalOptions = Readonly<{
122122

123123
type IndexedPlugin = Readonly<{
124124
// $FlowFixMe[unclear-type] Plugin types cannot be known statically
125-
plugin: FileMapPlugin<any, any>,
125+
plugin: FileMapPlugin<any, any, unknown>,
126126
dataIdx: ?number,
127127
}>;
128128
type InternalEnqueuedEvent = Readonly<
@@ -147,7 +147,9 @@ export type {FileDataPluginOptions} from './plugins/FileDataPlugin';
147147
export type {DependencyPluginOptions} from './plugins/DependencyPlugin';
148148
export {DuplicateHasteCandidatesError} from './plugins/haste/DuplicateHasteCandidatesError';
149149
export {HasteConflictsError} from './plugins/haste/HasteConflictsError';
150+
export {default as getPluginChanges} from './lib/getPluginChanges';
150151
export {default as HastePlugin} from './plugins/HastePlugin';
152+
export type {HasteChanges} from './plugins/HastePlugin';
151153

152154
export type {HasteMap} from './flow-types';
153155
export type {HealthCheckResult} from './Watcher';
@@ -318,19 +320,21 @@ export default class FileMap extends EventEmitter {
318320
const pluginWorkers: Array<FileMapPluginWorker> = [];
319321
const pluginDataIndices = new Map<string, number>();
320322
const plugins = options.plugins ?? [];
323+
const pluginNames = new Set<string>();
321324
for (const plugin of plugins) {
325+
// Plugin data slots, persisted plugin state and change summaries are all
326+
// addressed by name, so a second plugin of the same name would silently
327+
// take over the first's.
328+
invariant(
329+
!pluginNames.has(plugin.name),
330+
'metro-file-map: Duplicate plugin name: %s',
331+
plugin.name,
332+
);
333+
pluginNames.add(plugin.name);
322334
const maybeWorker = plugin.getWorker();
323335
const dataIdx = maybeWorker != null ? dataSlot++ : null;
324336
indexedPlugins.push({plugin, dataIdx});
325337
if (dataIdx != null) {
326-
// Crawlers address plugin data by name, so names must be unique among
327-
// plugins holding a slot - otherwise a crawler would silently write to
328-
// the shadowed plugin's slot.
329-
invariant(
330-
!pluginDataIndices.has(plugin.name),
331-
'metro-file-map: Duplicate plugin name: %s',
332-
plugin.name,
333-
);
334338
pluginDataIndices.set(plugin.name, dataIdx);
335339
}
336340
if (maybeWorker != null) {
@@ -507,7 +511,9 @@ export default class FileMap extends EventEmitter {
507511
]);
508512

509513
// Update `fileSystem` and plugins based on the file delta.
510-
const actualChanges = await this.#applyFileDelta(
514+
// Nothing is emitted for the initial build, so what plugins say about
515+
// it goes unused.
516+
const {changeAggregator: actualChanges} = await this.#applyFileDelta(
511517
fileSystem,
512518
plugins,
513519
fileDelta,
@@ -629,6 +635,27 @@ export default class FileMap extends EventEmitter {
629635
return null;
630636
}
631637

638+
/**
639+
* Bring every plugin up to date with a batch of changes, and collect what
640+
* each has to say about it for `ChangeEvent.pluginChanges`.
641+
*/
642+
#updatePlugins(
643+
changeAggregator: FileSystemChangeAggregator,
644+
): Map<string, unknown> {
645+
const pluginChanges = new Map<string, unknown>();
646+
this.#plugins.forEach(({plugin, dataIdx}) => {
647+
const summary = plugin.onChanged(
648+
changeAggregator.getMappedView(
649+
dataIdx != null ? metadata => metadata[dataIdx] : () => null,
650+
),
651+
);
652+
if (summary != null) {
653+
pluginChanges.set(plugin.name, summary);
654+
}
655+
});
656+
return pluginChanges;
657+
}
658+
632659
async #applyFileDelta(
633660
fileSystem: MutableFileSystem,
634661
plugins: ReadonlyArray<IndexedPlugin>,
@@ -637,7 +664,10 @@ export default class FileMap extends EventEmitter {
637664
removedFiles: ReadonlySet<CanonicalPath>,
638665
clocks?: WatchmanClocks,
639666
}>,
640-
): Promise<FileSystemChangeAggregator> {
667+
): Promise<{
668+
changeAggregator: FileSystemChangeAggregator,
669+
pluginChanges: ReadonlyMap<string, unknown>,
670+
}> {
641671
this.#startupPerfLogger?.point('applyFileDelta_start');
642672
const {changedFiles, removedFiles} = delta;
643673
this.#startupPerfLogger?.point('applyFileDelta_preprocess_start');
@@ -726,17 +756,11 @@ export default class FileMap extends EventEmitter {
726756
this.#startupPerfLogger?.point('applyFileDelta_add_end');
727757

728758
this.#startupPerfLogger?.point('applyFileDelta_updatePlugins_start');
729-
this.#plugins.forEach(({plugin, dataIdx}) => {
730-
plugin.onChanged(
731-
changeAggregator.getMappedView(
732-
dataIdx != null ? metadata => metadata[dataIdx] : () => null,
733-
),
734-
);
735-
});
759+
const pluginChanges = this.#updatePlugins(changeAggregator);
736760
this.#startupPerfLogger?.point('applyFileDelta_updatePlugins_end');
737761
this.#startupPerfLogger?.point('applyFileDelta_end');
738762

739-
return changeAggregator;
763+
return {changeAggregator, pluginChanges};
740764
}
741765

742766
/**
@@ -844,13 +868,7 @@ export default class FileMap extends EventEmitter {
844868
}
845869

846870
const _netChange = changeAggregator.getView();
847-
this.#plugins.forEach(({plugin, dataIdx}) => {
848-
plugin.onChanged(
849-
changeAggregator.getMappedView(
850-
dataIdx != null ? metadata => metadata[dataIdx] : () => null,
851-
),
852-
);
853-
});
871+
const pluginChanges = this.#updatePlugins(changeAggregator);
854872

855873
const toPublicMetadata = (
856874
metadata: Readonly<FileMetadata>,
@@ -877,6 +895,7 @@ export default class FileMap extends EventEmitter {
877895
const changeEvent: ChangeEvent = {
878896
changes: changesWithMetadata,
879897
logger: hmrPerfLogger,
898+
pluginChanges,
880899
rootDir: this.#options.rootDir,
881900
};
882901
this.emit('change', changeEvent);
@@ -1023,7 +1042,10 @@ export default class FileMap extends EventEmitter {
10231042
}
10241043

10251044
// Reuse the same batch processing logic as build()
1026-
const recrawlChangeAggregator = await this.#applyFileDelta(
1045+
const {
1046+
changeAggregator: recrawlChangeAggregator,
1047+
pluginChanges: recrawlPluginChanges,
1048+
} = await this.#applyFileDelta(
10271049
fileSystem,
10281050
this.#plugins,
10291051
crawlResult,
@@ -1051,6 +1073,7 @@ export default class FileMap extends EventEmitter {
10511073
const changeEvent: ChangeEvent = {
10521074
changes: changesWithMetadata,
10531075
logger: null,
1076+
pluginChanges: recrawlPluginChanges,
10541077
rootDir: this.#options.rootDir,
10551078
};
10561079
this.emit('change', changeEvent);

0 commit comments

Comments
 (0)