Repository navigation
Explore Rolldown tree-shaking of unused service methods - #10378
Closed
effect-bot wants to merge 8 commits into
Closed
effect-bot wants to merge 8 commits into
effect-bot wants to merge 8 commits into
Conversation
Fixtures cover a generator, combinators (use, flatMap, pipe), direct Context access, overrides with a scoped mock layer and updateService, and three bailouts (escape with computed access, Object.keys inspection, raw string key lookup). ServiceShaking.bundle is a passthrough seam, so the four pruning assertions fail until the plugin lands. Refs EFF-3380
ServiceShaking.bundle runs two Rolldown builds. The first records every transformed module; a key-level analysis then computes which methods are read from each service key, follows provider object literals through a bounded set of insertion shapes, and bails on any key use it does not recognise. The second build deletes undemanded provider properties and lets Rolldown drop the helpers only they referenced. APIs that hand a whole Context or a Fiber to untrusted code disable the optimization, since inspecting either prints every service value. FileSystem.make annotates its glob and stream constructors as pure so the derived methods can be removed. Refs EFF-3380
🦋 Changeset detectedLatest commit: 78da062 The changes in this PR will be included in the next version bump. This PR includes changesets to release 33 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Differential fixtures for each confirmed miscompilation: release and tap callbacks, Fiber exposure through Logger.make and onFiberStart (including a discarded runFork), receiver methods passed by identifier, inherited Object members, async factories, key mutation, a comment before a removed property's comma, an exported Layer built outside the bundle, and a consumer that transforms differently in the second build. All eleven fail against the current implementation. Refs EFF-3380
- Release and tap callbacks are consumers of the acquired provider object. - Untrusted code may only reference an audited allow-list of trusted APIs, replacing the Fiber/Context deny-list; run functions are only allowed without RunOptions, and runFork only when its Fiber is discarded. - Receiver reads are resolved for identifier-valued and spread-in methods; unresolvable receivers keep every property. - Inherited Object.prototype members observe the whole service. - A Layer or Context built from a provider must stay inside the bundle. - Async and generator functions are not followed as provider factories. - Writes to a service key's members, or untrusted key escapes, bail globally. - Property removal skips comments before the separator. - The second build is compared against every module of the first. Refs EFF-3380
Differential cases for confirmed miscompilations: code importing the bundle running inside a provider's scope (an Effect or Layer argument, an exported Effect.provide transformer, an exported Effect run with the importer's logger), the Redactable protocol and the global current Fiber exposing the Context, an external module's logger, callbacks reading arguments[0], a reassigned receiver function, nested provider removals, and an import resolving differently in the second build. The text-drift test now requires the first build's result. All eleven new cases fail against the current implementation. Refs EFF-3380
- The entry module may only export primitive constants, since importers can run code inside a provider's scope through arguments, returned Effects or exported transformers. - Untrusted references to the global object, eval/Function, Reflect, Proxy, require, Symbol.for and symbol-revealing Object APIs, and stack trace hooks, disable pruning: the running Fiber is published on the global object and Redactable hands formatters the current Context. - Imports left external, other than Node builtins, disable pruning. - Callbacks that read arguments are unknown consumers. - Reassigned functions are unresolved receivers. - The second build is also compared by resolved imports. - Edits nested in a removed property are dropped. Refs EFF-3380
…essions Fixtures other than the exported-entry cases no longer export main: they run as closed programs in a child process and print their result, so the entry-export bailout no longer masks the rule each one targets. Inspection code uses plain computed reads instead of Reflect for the same reason, and the external helper has no imports so every runtime can load it. Adds constructor-gen and constructor-map: providers whose methods are constructor-local consts, checking that writeOne and the demand only it had are removed while construction-time calls, service acquisition and release still run. Refs EFF-3380
Contributor
Author
|
Closing in favour of the native Rolldown implementation, which supersedes this plugin approach: effect-bot/rolldown#1 The Rolldown fork now holds the deliverable, its regression suite, its documentation and its own CI. This branch is no longer maintained. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proof of concept: a Rolldown build that deletes Effect service methods a program can't observe, with no Rolldown fork. For a closed
NodeRuntime.runMainprogram that only callsFileSystem.readFile, it drops every otherNodeFileSystemmethod, their helpers, and thenode:crypto/node:osimports, with identical output. Anything it can't prove safe falls back to the unoptimized bundle, which is most real programs today (see the recommendation).Closes EFF-3380
How it works
ServiceShaking.bundle(options)in@effect/bundle(private tooling package) runs two Rolldown builds:posttransform records every module.internal/analysis.tsthen computes, for each service key string, which methods are read.If any module transforms differently in the second build, the first build's output is returned. This covers every module, not just the edited ones.
Demand is keyed on the service key:
yield* K,K.use/useSync,Effect.map/flatMap/andThen/tap(K, f),K.pipe(...),Effect.service(K),Context.get*(ctx, K), and theEffect.updateServicecallback. The value may only be read through static members or destructuring, or spread into another provider literal. Reading an inheritedObject.prototypemember (valueOf,hasOwnProperty, ...) bails the key.Layer.succeed/effect/sync,Effect.provideService(Effect),Effect.updateService,K.context,Context.make/add, andK.of. A literal is pruned only when a bounded flow proves three things:Effect.succeed/sync/map/flatMap/tap/acquireRelease/gen, pipes, single-use consts, and non-escaping synchronous functions. Async and generator functions are not followed.tap) only reads static members.Effect.provide,Layer.launch, theLayer.merge*/provide*combinators, consts, and factory functions whose calls are followed in turn. Reaching an entry export or anything else bails the key.{ ...impl }passes demand on to argument literals, plus static reads of the parameter.thisreads are demanded on every object that can hold them, including objects the method is spread into. A method that can't be resolved keeps every property.Bailouts
Per key: the service value escapes, the key is used in an unrecognised way, the key is created outside a declaration, or the Layer holding a provider escapes. A callback that reads
arguments, or a receiver function that is reassigned, counts as unknown.Global (nothing is pruned):
toStringon objects passed as "data"), returned Effects, or exported transformers. None of that can be checked from inside the bundle.allowedApis/allowedModules. A Fiber exposes its Context as a public property, and a Context hands out any service by key string:Context.getUnsafewith a rebuiltContext.Service("same key"),mapUnsafe, or traversal fromObject.values(fiber). So APIs that hand out a Fiber or Context must bail. That includes nested callback types such asLogger.Options.fiberandRunOptions.onFiberStart. Run functions are only allowed without options, andrunForkonly when its result is discarded.globalThis["~effect/Fiber/currentFiber"].Redactablepasses the current Context to any object holdingSymbol.for("~effect/Redactable"), whichOption'stoStringreaches throughFormatter. So untrusted references to these bail:globalThis/global/self/window,eval/Function,Reflect,Proxyandrequire;Symbolbeyonditerator/asyncIterator;ObjectAPIs;.context/.mapUnsafereads, dynamic keys, namespaces used as values, dynamicimport(), and writes to a key's members.The second build must match the first in every module's code, every resolved import, and the module set. Otherwise the first build's output is returned. Edits nested inside a removed property are dropped.
Assumptions (not checked):
effectand@effect/platform-*don't inspect service values or hand out a Context or Fiber except through the APIs above, and the global built-ins aren't modified by code outside the bundle.On printing
Printing a Context or Fiber with
console.log/util.inspect,Formatter.formatorEffect.logcurrently lists each service's property names (unused: [Function: unused]).JSON.stringifyandString(fiber)don't. If printed service shapes are not treated as program output, printing alone is no reason to bail. None of the remaining bailouts depends on printing: each regression fixture retrieves a service explicitly and callstypeofon a method.Results
Same build for every column (workspace sources, annotate-pure-calls, unminified). Closed-world probes run as
NodeRuntime.runMainprograms, with stdout compared through a child process.runMain+NodeFileSystem+readFilerunMain+NodeServices+readFile+Effect.logrunMain+ JSON + computed property accessrunMain+ a Schema decoderunMain+forkChild+Fiber.joinexport const main)Each
bundlecall takes about 0.3–0.8 s.Required Effect change
FileSystem.makemarksGlob.make(impl)andEffect.fnUntraced(...)as/* @__PURE__ */(patch changeset). Without the marks,globpassesimplwhole and every Node method stays.Recommendation
General use: negative. Narrow closed-world subset: positive. Rolldown needs no fork, and closed
runMainapplications using only audited APIs lose 14–41% of the bundle with identical output. But:Fiber.join/await/interrupt. That's an analysis expansion I haven't done.The larger lever is on the Effect side: services whose methods can't be reached by key string or through the current Fiber, or explicit per-method services.
Tests
The four size tests (
... drops FileSystem methods ...) now fail by design: their fixtures exportmain, which the entry-point rule rejects. Every other test passes on Node, Bun and Deno (36/40).With the entry rule disabled locally, those four pass, and every regression passes through its own fix except the four
scope-*cases, which need the entry rule.Needed in a separate run:
export main, as closed-world entries run in a child process. This lets each fix be exercised on its own.Object.values(fiber)and rebuilt-key retrieval paths;NodeServicessmoke test.