Issue with nodejs_compat + dynamic require of node:events #854
Comments
dja
changed the title
|
Same exact issue for me as well using same package |
|
Having the same issue with trying to use This was with Wrangler version v.2.20.0 as well as v3.0.0. Also encountering this error with |
|
Anyone have any luck resolving this? |
|
Right now, the only way to access these compatible libraries is by a static |
|
I'm using |
|
Which workers syntax are y'all using? The older "server worker" syntax (using |
|
@jasnell here's useless but functional barebones example where I see the problem: worker.ts wrangler.toml package.json I haven't deployed this to the actual worker instance, but I see the issue I was seeing before with miniflare locally: |
|
@dcrodman — looks like this issue is specific to the legacy service worker syntax. I can reproduce it, but only when using the addEventListener('fetch', (event) => {
let fetchEvent: FetchEvent = event as FetchEvent;
fetchEvent.respondWith(handle(fetchEvent));
});Instead, please use the ESM syntax: export default {
async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
let b = Buffer.from(request.body);
console.log(b);
return new Response('Hello World!');
},
};I'll make sure to update our docs to be clearer about this, and we'll take a look at our error messages in this case. For more on ESM syntax, and to migrate from using service workers, refer to: https://developers.cloudflare.com/workers/learning/migrate-to-module-workers/ |
|
Thank you @irvinebroque , the docs update feels like the most important part here since this can create a potentially difficult situation for people trying to upgrade to Wrangler 3. Migrating is still not necessarily easy and knowing this up front would've been super helpful. |
|
I think my statement above was misleading and not quite right. Simple reproduction of the issue: https://github.com/irvinebroque/nodejs_compat_issue/blob/main/src/worker.ts // This does not work
const { Writable } = require('node:stream');
// This works
// import { Writable } from "node:stream";
const myStream = new Writable();
export default {
async fetch(request) {
return new Response("hello");
},
};We will take a look at what we can do here. |
|
|
|
The issue still persists in the latest |
|
@irvinebroque ... the |
|
Ah right, unless the userland code is uploaded as a Node.js module type (#564, only supported in workerd, not by surrounding tooling) or CommonJS module type (only used if manually configured AFAIK) — then you can't use But that because Wrangler runs esbuild, it effectively papers over this — it looks like Scaffolding out some better docs in cloudflare/cloudflare-docs#13344 |
|
I'm going to remove the |
jasnell
added
feature request
jasnell
changed the title
|
Does anyone have a solution ? |
|
I am having this issue trying to run cheerio :(
|
|
How to solve this? I encountered |
|
You can't. You have to either use a different platform or rewrite the original library's code to not use that part. Hopefully it is not needed. I am not sure why cfw has these limits that cripple most NPM... why they cannot just run Bun.js in Alpine? |
|
I fixed my I'm using vite + sveltekit + cloudflare pages |
|
@xcaptain , can you share an example or POC of your solution? I'm also fighting with Sveltekit and cloudflare pages :/ |
|
Yeah when I do that vite-plugin-node-polyfills and add it to vite config then it gives me even worse error in Bun.js with wrangler 3.57 (newer versions also cause stack error) ` <--- Last few GCs ---> [8792:000001DF409FF5D0] 254870 ms: Scavenge (reduce) 4069.2 (4142.8) -> 4069.0 (4143.5) MB, 3273.80 / 0.00 ms (average mu = 0.603, current mu = 0.411) allocation failure; <--- JS stacktrace ---> FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory 1: 00007FF6FF7A39EB node::SetCppgcReference+18459 |
You can check my demo repo at: https://github.com/xcaptain/cf-demo1/blob/main/src/routes/test/%2Bserver.ts#L2 Run: bun i
bun run build
bunx wrangler pages dev .\.svelte-kit\cloudflareThen: Would see error: The reason is var buffer = require('buffer')require is not found in cloudflare worker. My solution is to not use |
|
@xcaptain - the problem we have for your example is that the Svelte Cloudflare adaptor is bundling the generated worker code into a single file using this esbuild call: https://github.com/sveltejs/kit/blob/ed6b5cd4fc0d1d299c838628b73027dd07179281/packages/adapter-cloudflare/index.js#L87-L109 const result = await esbuild.build({
platform: 'browser',
// https://github.com/cloudflare/workers-sdk/blob/a12b2786ce745f24475174bcec994ad691e65b0f/packages/wrangler/src/deployment-bundle/bundle.ts#L35-L36
conditions: ['workerd', 'worker', 'browser'],
sourcemap: 'linked',
target: 'es2022',
entryPoints: [`${tmp}/_worker.js`],
outfile: `${dest}/_worker.js`,
allowOverwrite: true,
format: 'esm',
bundle: true,
loader: {
'.wasm': 'copy',
'.woff': 'copy',
'.woff2': 'copy',
'.ttf': 'copy',
'.eot': 'copy',
'.otf': 'copy'
},
external,
alias: Object.fromEntries(compatible_node_modules.map((id) => [id, `node:${id}`])),
logLevel: 'silent'
});What this does is treat all the node module built in imports as external. As such it doesn't attempt to inline the var __require = /* @__PURE__ */ ((x) => typeof require !== "undefined" ? require : typeof Proxy !== "undefined" ? new Proxy(x, {
get: (a, b) => (typeof require !== "undefined" ? require : a)[b]
}) : x)(function(x) {
if (typeof require !== "undefined") return require.apply(this, arguments);
throw Error('Dynamic require of "' + x + '" is not supported');
});This is where the runtime exceptions appear. |
|
The good news is that as of 2024-09-23 (compat date), we have a new version of The output from the name = "cf-demo1"
compatibility_date = "2024-10-04"
compatibility_flags = ["nodejs_compat"]
assets = { directory = ".svelte-kit/output/client" }
main = ".svelte-kit/cloudflare-tmp/_worker.js"
[vars]
BUN_VERSION = "1.1"Now running |
|
Hi, i add the same problem with Cloudflare Pages and sveltkit adapter, here is a small patch that seems to do the trick: diff --git a/index.js b/index.js
index 0fe55898e5697fc6a061299780e163ca4553cb05..0bd37d9fda86340ac77585e1334707b2cf0c5c48 100644
--- a/index.js
+++ b/index.js
@@ -81,7 +81,7 @@ export default function (options = {}) {
}
});
- const external = ['cloudflare:*', ...compatible_node_modules.map((id) => `node:${id}`)];
+ const external = ['cloudflare:*', /*...compatible_node_modules.map((id) => `node:${id}`)*/];
try {
const result = await esbuild.build({
@@ -104,7 +104,7 @@ export default function (options = {}) {
'.otf': 'copy'
},
external,
- alias: Object.fromEntries(compatible_node_modules.map((id) => [id, `node:${id}`])),
+ //alias: Object.fromEntries(compatible_node_modules.map((id) => [id, `node:${id}`])),
logLevel: 'silent'
});
|
|
The issue with my imports was resolved by adding the following to the The reason for the problem was that the |
|
@petebacondarwin I'm having the same problem and even after updating the @JJK801 can you please explain a bit more on how to apply this patch? |
|
@xcaptain what's your experience working with solana web3.js v2? I am relying on metaplex umi & mpl-core, so I can't switch to web3.js v2. |
Which Cloudflare product(s) does this pertain to?
Workers/Other, Wrangler
What version of
Wranglerare you using?2.18.0
What operating system are you using?
Mac
Describe the Bug
Attempting to upload a worker with Wrangler using
compatibility_flags = ['nodejs_compat']or command line--compatibility-flags="nodejs_compat"fails with this error message:The package using
node:eventsis this LaunchDarkly SDK: https://github.com/launchdarkly/js-core/tree/main/packages/sdk/cloudflareThe text was updated successfully, but these errors were encountered: