Segmentation fault on require("@swc/core") in a Node.js worker thread #2276

andersk · 2021-09-20T22:26:49Z

Running require("@swc/core") inside a Node.js worker thread causes a segfault. (This causes problems with terser-webpack-plugin’s new swcMinify function.)

// test.js
const { Worker } = require("worker_threads");
new Worker('require("@swc/core")', { eval: true });

$ node test.js
Segmentation fault (core dumped)

I’m running Node 14.17.6 on Linux x86-64 (NixOS 21.11). In case this is system-dependent, here’s a Dockerfile that should reproduce reliably:

# Dockerfile
FROM node:14.17.6
WORKDIR /tmp
RUN npm i @swc/[email protected]
RUN node -e 'new worker_threads.Worker(`require("@swc/core")`, { eval: true })'

$ docker build .
Sending build context to Docker daemon  57.85MB
Step 1/4 : FROM node:14.17.6
 ---> 9dc0768a6295
Step 2/4 : WORKDIR /tmp
 ---> Using cache
 ---> ade3fef59802
Step 3/4 : RUN npm i @swc/[email protected]
 ---> Using cache
 ---> e1b2d22eae67
Step 4/4 : RUN node -e 'new worker_threads.Worker(`require("@swc/core")`, { eval: true })'
 ---> Running in 25299ad85030
Segmentation fault (core dumped)
The command '/bin/sh -c node -e 'new worker_threads.Worker(`require("@swc/core")`, { eval: true })'' returned a non-zero code: 139

The text was updated successfully, but these errors were encountered:

kdy1 · 2021-09-21T07:29:30Z

Native node bindings are expected to

Call async operations from js thread

or

Call synchronous function from worker thread

It's the limitation of node js itself.

cc @Brooooooklyn
Am I right?

andersk · 2021-09-21T19:51:09Z

I don’t know of any such limitation. Perhaps you’re confusing worker threads (JS threads created using the worker_threads API) with threads in the worker pool (native threads created from addons to perform synchronous operations that are exposed to JS via a custom asynchronous JS API)?

In worker threads, Node.js addons should be able to perform synchronous and asynchronous operations, provided they follow certain rules about thread safety and context awareness.

Brooooooklyn · 2021-09-22T03:07:26Z

In worker threads, Node.js addons should be able to perform synchronous and asynchronous operations, provided they follow certain rules about thread safety and context awareness.

Yes. I will dig into this issue today.

Brooooooklyn · 2021-09-23T05:13:12Z

This issue only occurs on Linux GNU platform with napi-rs. I've tried Linux musl / macOS / Windows, all these platform is fine.
And I've also tried this example https://github.com/nodejs/node-addon-examples/tree/main/async_pi_estimate/node-addon-api on Linux GNU, also works fine.

I need more time to dig into it.

kdy1 · 2021-12-16T06:15:35Z

Does it happen only on centos?
Maybe it's a bug of glibc in centos 7

andersk · 2021-12-16T17:30:28Z

@kdy1 I originally found this on NixOS 21.11; the node:14.17.6 Docker image is based on Debian 9, and it also reproduces with node:16.13.1 which is based on Debian 10. Since you asked, I was also able to reproduce on CentOS 8.4.2105. As far as I can tell, all glibc platforms are impacted. (Did you find a glibc platform that isn’t?)

Brooooooklyn · 2021-12-17T03:32:46Z

Maybe a valid workaround rust-lang/rust#91979 (comment)

Jackson-p · 2022-01-24T06:05:56Z

+1,i meet this problem too,it also caused by terser-webpack-plugin’s new swcMinify function.

Brooooooklyn · 2022-01-24T07:47:19Z

@Jackson-p What are your system and Glibc versions

Jackson-p · 2022-01-24T08:05:48Z

@Jackson-p What are your system and Glibc versions

CentOS Linux release 7.4.1708(Core)

ldd(GNU libc) 2.17

@Brooooooklyn

Brooooooklyn · 2022-01-24T10:18:21Z

I've confirmed, this issue should have been fixed in #3314

docker run -w /root --rm -it node:14.17.6 sh -c "npm i @swc/core && node -e 'new worker_threads.Worker(\`console.log(require(\"@swc/core\"))\`, { eval: true })'"
npm WARN saveError ENOENT: no such file or directory, open '/root/package.json'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: @swc/[email protected] (node_modules/@swc/core/node_modules/@swc/core-win32-ia32-msvc):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for @swc/[email protected]: wanted {"os":"win32","arch":"ia32"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: @swc/[email protected] (node_modules/@swc/core/node_modules/@swc/core-win32-x64-msvc):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for @swc/[email protected]: wanted {"os":"win32","arch":"x64"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: @swc/[email protected] (node_modules/@swc/core/node_modules/@swc/core-linux-arm-gnueabihf):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for @swc/[email protected]: wanted {"os":"linux","arch":"arm"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: @swc/[email protected] (node_modules/@swc/core/node_modules/@swc/core-linux-arm64-musl):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for @swc/[email protected]: wanted {"os":"linux","arch":"arm64"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: @swc/[email protected] (node_modules/@swc/core/node_modules/@swc/core-android-arm64):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for @swc/[email protected]: wanted {"os":"android","arch":"arm64"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: @swc/[email protected] (node_modules/@swc/core/node_modules/@swc/core-darwin-arm64):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for @swc/[email protected]: wanted {"os":"darwin","arch":"arm64"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: @swc/[email protected] (node_modules/@swc/core/node_modules/@swc/core-linux-arm64-gnu):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for @swc/[email protected]: wanted {"os":"linux","arch":"arm64"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: @swc/[email protected] (node_modules/@swc/core/node_modules/@swc/core-win32-arm64-msvc):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for @swc/[email protected]: wanted {"os":"win32","arch":"arm64"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: @swc/[email protected] (node_modules/@swc/core/node_modules/@swc/core-darwin-x64):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for @swc/[email protected]: wanted {"os":"darwin","arch":"x64"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: @swc/[email protected] (node_modules/@swc/core/node_modules/@swc/core-android-arm-eabi):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for @swc/[email protected]: wanted {"os":"android","arch":"arm"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: @swc/[email protected] (node_modules/@swc/core/node_modules/@swc/core-freebsd-x64):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for @swc/[email protected]: wanted {"os":"freebsd","arch":"x64"} (current: {"os":"linux","arch":"x64"})
npm WARN enoent ENOENT: no such file or directory, open '/root/package.json'
npm WARN root No description
npm WARN root No repository field.
npm WARN root No README data
npm WARN root No license field.

+ @swc/[email protected]
added 3 packages from 1 contributor and audited 14 packages in 68.157s

1 package is looking for funding
  run `npm fund` for details

found 0 vulnerabilities

{
  version: '1.2.133',
  plugins: [Function: plugins],
  Compiler: [class Compiler],
  parse: [Function: parse],
  parseSync: [Function: parseSync],
  parseFile: [Function: parseFile],
  parseFileSync: [Function: parseFileSync],
  print: [Function: print],
  printSync: [Function: printSync],
  transform: [Function: transform],
  transformSync: [Function: transformSync],
  transformFile: [Function: transformFile],
  transformFileSync: [Function: transformFileSync],
  bundle: [Function: bundle],
  minify: [Function: minify],
  minifySync: [Function: minifySync],
  DEFAULT_EXTENSIONS: [
    '.js',  '.jsx',
    '.es6', '.es',
    '.mjs', '.ts',
    '.tsx'
  ]
}

Brooooooklyn · 2022-01-24T10:21:37Z

/cc @devongovett since you can use the zig as cross compiling linker to support GLIBC < 2.12, you can avoid the __cxa_thread_atexit_impl issue in rust-lang/rust#91979 . So no more workaround is needed.

kdy1 · 2022-01-24T10:38:46Z

@Brooooooklyn Do you mean this issue can be closed?

Brooooooklyn · 2022-01-24T10:41:04Z

Do you mean this issue can be closed?

Yes

kdy1 · 2022-01-24T13:51:36Z

Thank you!

devongovett · 2022-01-24T15:29:50Z

@Brooooooklyn could you explain how this works and how it fixes the Rust issue? Does this mean it doesn't depend on glibc somehow?

Brooooooklyn · 2022-01-25T03:42:42Z

@devongovett It is a little complex, I'll write an article to talk about this.

In a nutshell, __cxa_thread_atexit_impl is introduced in std/src/sys/unix/thread_local_dtor.rs#L7-L13

But it's weakly linked since Rust std has a fallback implementation: std/src/sys/unix/thread_local_dtor.rs#L27

If a library is built with target < GLIBC_2.18, the __cxa_thread_atexit_impl symbol will not be linked.

devongovett · 2022-01-25T04:31:24Z

@Brooooooklyn ah interesting. So if I understand correctly, another solution would be to build on a system (or docker image) with an older glibc version? What kind of magic is zig doing here?

I'll write an article to talk about this.

That would be great!

Brooooooklyn · 2022-01-25T05:17:54Z

So if I understand correctly, another solution would be to build on a system (or docker image) with an older glibc version?

Yes, we can only compile our codes on an older glibc version system without zig.

If we are using zig as the program linker, we can target our binary to whatever glibc versions, like x86_64-linux-gnu.2.12. So we can build a binary which compatible with an older glibc system on a modern system.

https://andrewkelley.me/post/zig-cc-powerful-drop-in-replacement-gcc-clang.html
https://github.com/napi-rs/napi-rs/blob/main/cli/src/build.ts#L279

swc-bot · 2022-10-19T00:37:20Z

This closed issue has been automatically locked because it had no new activity for a month. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.

andersk added the C-bug label Sep 20, 2021

kdy1 closed this as completed Jan 24, 2022

kdy1 added this to the v1.2.133 milestone Jan 24, 2022

yisibl mentioned this issue Mar 14, 2022

Update @resvg/resvg-js to v2.0.0 svg-sprite/svg-sprite#606

Merged

swc-project locked as resolved and limited conversation to collaborators Oct 19, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Segmentation fault on require("@swc/core") in a Node.js worker thread #2276

Segmentation fault on require("@swc/core") in a Node.js worker thread #2276

andersk commented Sep 20, 2021 •

edited

Loading

kdy1 commented Sep 21, 2021

andersk commented Sep 21, 2021

Brooooooklyn commented Sep 22, 2021

Brooooooklyn commented Sep 23, 2021

kdy1 commented Dec 16, 2021

andersk commented Dec 16, 2021 •

edited

Loading

Brooooooklyn commented Dec 17, 2021

Jackson-p commented Jan 24, 2022

Brooooooklyn commented Jan 24, 2022

Jackson-p commented Jan 24, 2022

Brooooooklyn commented Jan 24, 2022

Brooooooklyn commented Jan 24, 2022

kdy1 commented Jan 24, 2022

Brooooooklyn commented Jan 24, 2022

kdy1 commented Jan 24, 2022

devongovett commented Jan 24, 2022

Brooooooklyn commented Jan 25, 2022

devongovett commented Jan 25, 2022

Brooooooklyn commented Jan 25, 2022 •

edited

Loading

swc-bot commented Oct 19, 2022

Segmentation fault on require("@swc/core") in a Node.js worker thread #2276

Segmentation fault on require("@swc/core") in a Node.js worker thread #2276

Comments

andersk commented Sep 20, 2021 • edited Loading

kdy1 commented Sep 21, 2021

andersk commented Sep 21, 2021

Brooooooklyn commented Sep 22, 2021

Brooooooklyn commented Sep 23, 2021

kdy1 commented Dec 16, 2021

andersk commented Dec 16, 2021 • edited Loading

Brooooooklyn commented Dec 17, 2021

Jackson-p commented Jan 24, 2022

Brooooooklyn commented Jan 24, 2022

Jackson-p commented Jan 24, 2022

Brooooooklyn commented Jan 24, 2022

Brooooooklyn commented Jan 24, 2022

kdy1 commented Jan 24, 2022

Brooooooklyn commented Jan 24, 2022

kdy1 commented Jan 24, 2022

devongovett commented Jan 24, 2022

Brooooooklyn commented Jan 25, 2022

devongovett commented Jan 25, 2022

Brooooooklyn commented Jan 25, 2022 • edited Loading

swc-bot commented Oct 19, 2022

andersk commented Sep 20, 2021 •

edited

Loading

andersk commented Dec 16, 2021 •

edited

Loading

Brooooooklyn commented Jan 25, 2022 •

edited

Loading