GH-49995: [R][Wasm] Fix threading issue under Wasm/Emscripten by amoeba · Pull Request #49996 · apache/arrow

amoeba · 2026-05-19T21:17:17Z

Rationale for this change

#49982 added a basic Wasm build but after doing some testing I ran into an issue with threading.

What changes are included in this PR?

The main fix: Modified CanRunWithCapturedR so it's always false under Emscripten. This change is the only change that was strictly needed to fix my issue. I'm not a pro with this part of the codebase so scrutiny welcome.
I also set options(arrow.use_threads = FALSE) when running under Wasm just in case since it makes sense to not use threads in a Wasm context
Added some skips to some tests, mostly timezone-related ones, so we skip some tests under Emscripten

Are these changes tested?

Yes. Crossbow jobs show the build works and I have a demo at https://amoeba.github.io/arrow-r-wasm/.

Are there any user-facing changes?

No.

GitHub Issue: [R][Wasm] Fix Error: thread constructor failed: Not supported under Wasm #49995

amoeba · 2026-05-19T21:17:33Z

@github-actions crossbow submit test-r-wasm

github-actions · 2026-05-19T21:19:52Z

Revision: dadf0f5

Submitted crossbow builds: ursacomputing/crossbow @ actions-9b0263bfa9

Task	Status
test-r-wasm

amoeba · 2026-05-19T21:45:44Z

@github-actions crossbow submit test-r-wasm

amoeba · 2026-05-19T21:46:30Z

With dadf0f5, Parquet reading works with my local test. I'm testing the changes separately so I know what's actually fixing things here.

github-actions · 2026-05-19T21:47:55Z

Revision: abaacb8

Submitted crossbow builds: ursacomputing/crossbow @ actions-510511a107

Task	Status
test-r-wasm

amoeba · 2026-05-20T03:57:29Z

I marked this as ready for review. The PR description hopefully explains all that's needed and I put together a demo at https://amoeba.github.io/arrow-r-wasm/

jonkeane · 2026-05-20T12:31:01Z

+bool CanRunWithCapturedR() {
+#ifdef __EMSCRIPTEN__
+  // Threading is not supported under Emscripten/WASM. Always take the
+  // synchronous path to avoid attempting pthread_create which will fail
+  // with "thread constructor failed: Not supported".
+  return false;
+#else
+  return MainRThread::GetInstance().Executor() == nullptr;
+#endif


Is it possible to add a test (or maybe it's that we actually need to run the tests under wasm(???) to catch this?

It looks like maybe we can at least run some R code in the container if not run the test suite. I'll report back.

amoeba · 2026-05-20T19:40:10Z

@github-actions crossbow submit test-r-wasm

amoeba · 2026-05-20T19:41:21Z

I'm testing trying to install, load the package, and run the test suite in CI here. I tried testing locally but I'm running into what appears to be some qemu errors since I'm running Docker on Mac so we'll see how this runs in CI.

github-actions · 2026-05-20T19:42:41Z

Revision: 19efaf3

Submitted crossbow builds: ursacomputing/crossbow @ actions-6a9811d919

Task	Status
test-r-wasm

amoeba · 2026-05-20T23:28:29Z

@github-actions crossbow submit test-r-wasm

github-actions · 2026-05-20T23:30:44Z

Revision: 47738fa

Submitted crossbow builds: ursacomputing/crossbow @ actions-8e088c6ac8

Task	Status
test-r-wasm

amoeba · 2026-05-20T23:58:48Z

The job runs but we don't actually run any tests,

Running testthat suite under webR...
No installed testthat tests found for arrow.
Results: 0 passed, 0 skipped, 0 failed, 0 errors
✓ testthat suite passed
✓ All tests passed!

More soon!

amoeba · 2026-05-21T03:33:26Z

@github-actions crossbow submit test-r-wasm

github-actions · 2026-05-21T03:36:06Z

Revision: 9f03a2a

Submitted crossbow builds: ursacomputing/crossbow @ actions-d71cf926ef

Task	Status
test-r-wasm

amoeba · 2026-05-21T03:42:04Z

I couldn't find any examples of people running their test suites under Wasm but I'm going to see if I can't get it working in 9f03a2a. If that doesn't work easily, I think it'd be good enough to just write a simple test script to run instead.

amoeba · 2026-05-21T03:55:02Z

That went shockingly well,

Results: 6376 passed, 92 skipped, 1 failed, 3 errors

Here are the errors/failures:

── 1. Error ('test-dataset-dplyr.R:76:3'): filter() on timestamp columns ───────
Error in `compute.arrow_dplyr_query(x)`: Invalid: Cannot locate or parse timezone 'UTC': discover_tz_dir failed to find zoneinfo

── 2. Error ('test-dataset-dplyr.R:124:3'): filter() on date32 columns ─────────
Error in `compute.arrow_dplyr_query(x)`: Invalid: Cannot locate or parse timezone 'UTC': discover_tz_dir failed to find zoneinfo

── 3. Failure ('test-dplyr-filter.R:425:3'): filter() with across() ────────────
Expected `via_table <- rlang::eval_tidy(expr, rlang::new_data_mask(rlang::env(.input = arrow_table(tbl))))` not to throw any warnings.

── 4. Error ('test-dplyr-query.R:650:3'): Scalars in expressions match the type 
Error: Error: NotImplemented: Function 'greater' has no kernel matching input types (timestamp[us, tz=UTC], string)

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

+  const server = http.createServer((req, res) => {
+    const filePath = path.join(repoDir, decodeURIComponent(req.url));
+    if (!filePath.startsWith(path.resolve(repoDir))) {
+      res.writeHead(403);
+      res.end();
+      return;
+    }


    example_data
  )
+  skip_on_emscripten()
  compare_dplyr_binding(


+        tz_path <- tzdb::tzdb_path("text")
+        packageStartupMessage("[configure_tzdb] tzdb path: ", tz_path)
+        packageStartupMessage("[configure_tzdb] path exists: ", dir.exists(tz_path))
+        if (dir.exists(tz_path)) {
+          tz_files <- list.files(tz_path, recursive = TRUE)


-bool CanRunWithCapturedR() { return MainRThread::GetInstance().Executor() == nullptr; }
+bool CanRunWithCapturedR() {
+#ifdef __EMSCRIPTEN__
+  // Threading is not supported under Emscripten/WASM. Always take the


Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 7 comments.

+        tz_path <- tzdb::tzdb_path("text")
+        packageStartupMessage("[configure_tzdb] tzdb path: ", tz_path)
+        packageStartupMessage("[configure_tzdb] path exists: ", dir.exists(tz_path))
+        if (dir.exists(tz_path)) {
+          tz_files <- list.files(tz_path, recursive = TRUE)


        packageStartupMessage(
-          "The tzdb package was available but failed to initialize: ",
-          e,
+          "[configure_tzdb] tzdb package available but failed to initialize: ",
+          conditionMessage(e)
+        )


    packageStartupMessage(
-      "The tzdb package is not installed. ",
+      "[configure_tzdb] tzdb package is NOT installed. ",
      "Timezones will not be available to Arrow compute functions. ",
      "If you get errors when using Arrow on datetimes, try running ",
      "`install.packages('tzdb')` and trying again."


+  const server = http.createServer((req, res) => {
+    const filePath = path.join(repoDir, decodeURIComponent(req.url));
+    if (!filePath.startsWith(path.resolve(repoDir))) {
+      res.writeHead(403);
+      res.end();
+      return;
+    }
+    fs.readFile(filePath, (err, data) => {


+tgz_file=$(ls "${arrow_r_dir}"/arrow_*.tgz 2>/dev/null | head -1)
+if [ -z "${tgz_file}" ]; then
+  echo "ERROR: No arrow_*.tgz found in ${arrow_r_dir}" >&2
+  exit 1
+fi


      collect(),
    example_data
  )
+  skip_on_emscripten()


  skip_if_not_available("re2")
+  skip_on_emscripten()


amoeba · 2026-07-02T03:01:17Z

@github-actions crossbow submit test-r-wasm

github-actions · 2026-07-02T03:03:31Z

Revision: 4bf5669

Submitted crossbow builds: ursacomputing/crossbow @ actions-4d03e485b0

Task	Status
test-r-wasm

amoeba · 2026-07-02T03:36:52Z

@github-actions crossbow submit test-r-wasm

Copilot

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.

        packageStartupMessage(
          "The tzdb package was available but failed to initialize: ",
          e,
-          "Timezones will not be available to Arrow compute functions."
        )
      }


+arrow_r_dir="${1:-.}"
+
+# Set up a fake CRAN-like repo so we can install the package
+tgz_file=$(ls "${arrow_r_dir}"/arrow_*.tgz 2>/dev/null | head -1)


+  const server = http.createServer((req, res) => {
+    const filePath = path.join(repoDir, decodeURIComponent(req.url));
+    if (!filePath.startsWith(path.resolve(repoDir))) {
+      res.writeHead(403);
+      res.end();
+      return;
+    }


amoeba · 2026-07-02T06:38:25Z

@github-actions crossbow submit test-r-wasm

pitrou · 2026-07-02T08:08:44Z

Hey @DerThorsten do you want to take a look at this? You're probably one of the world experts on this :)

DerThorsten · 2026-07-02T08:12:08Z

Hey @DerThorsten do you want to take a look at this? You're probably one of the world experts on this :)

Sure, but not sure if I manage today

pitrou · 2026-07-02T08:13:00Z

There's no urgency :)

amoeba · 2026-07-02T17:12:26Z

@github-actions crossbow submit test-r-wasm

github-actions · 2026-07-02T17:14:39Z

Revision: 9cc1264

Submitted crossbow builds: ursacomputing/crossbow @ actions-83127ccf1f

Task	Status
test-r-wasm

rok · 2026-07-03T16:04:49Z

@amoeba you might want to rebase on this to get the test-r-wasm job passing the c++ compilation (it's been failing for 11 days now on nightly).

rok · 2026-07-03T16:11:10Z

Ah, I see you fixed the ncores issue in r/tools/nixlibs.R, never mind.

github-actions Bot added awaiting review Awaiting review Component: R labels May 19, 2026

amoeba marked this pull request as ready for review May 20, 2026 03:56

amoeba requested review from jonkeane and thisisnic as code owners May 20, 2026 03:56

amoeba changed the title ~~GH-49995: [R][Wasm] Fix threading issue under Wasm/Emscritpen~~ GH-49995: [R][Wasm] Fix threading issue under Wasm/Emscripten May 20, 2026

jonkeane reviewed May 20, 2026

View reviewed changes

github-actions Bot added awaiting changes Awaiting changes and removed awaiting review Awaiting review labels May 20, 2026

amoeba requested review from assignUser, kou and raulcd as code owners May 20, 2026 19:39

github-actions Bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels May 20, 2026

amoeba added 4 commits July 1, 2026 19:43

Make r_wasm_test.cjs slightly safer

0eb079b

respect force_tests in skip_on_emscripten

03e9002

Remove TODO

f309ad9

revert our changes to build_arrow_static.sh

3866da3

amoeba force-pushed the GH-49995--r-wasm-threading branch from 984f37e to 3866da3 Compare July 2, 2026 02:44

Copilot AI reviewed Jul 2, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings July 2, 2026 02:45

Copilot started reviewing on behalf of amoeba July 2, 2026 02:45 View session

Copilot AI reviewed Jul 2, 2026

View reviewed changes

amoeba added 2 commits July 1, 2026 19:51

revert debug messages in arrow-package.R

172c0d4

try to fix tz issue under wasm

4bf5669

amoeba requested a review from pitrou as a code owner July 2, 2026 03:01

github-actions Bot added the Component: C++ label Jul 2, 2026

Fix cmake issue from nixlibs

093a443

Copilot AI review requested due to automatic review settings July 2, 2026 03:36

Copilot started reviewing on behalf of amoeba July 2, 2026 03:36 View session

Copilot AI reviewed Jul 2, 2026

View reviewed changes

amoeba marked this pull request as draft July 2, 2026 03:42

Syntax fix

9cc1264

Uh oh!

Conversation

amoeba commented May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

amoeba commented May 19, 2026

Uh oh!

github-actions Bot commented May 19, 2026

Uh oh!

amoeba commented May 19, 2026

Uh oh!

amoeba commented May 19, 2026

Uh oh!

github-actions Bot commented May 19, 2026

Uh oh!

amoeba commented May 20, 2026

Uh oh!

jonkeane May 20, 2026

Choose a reason for hiding this comment

Uh oh!

amoeba May 20, 2026

Choose a reason for hiding this comment

Uh oh!

amoeba commented May 20, 2026

Uh oh!

amoeba commented May 20, 2026

Uh oh!

github-actions Bot commented May 20, 2026

Uh oh!

amoeba commented May 20, 2026

Uh oh!

github-actions Bot commented May 20, 2026

Uh oh!

amoeba commented May 20, 2026

Uh oh!

amoeba commented May 21, 2026

Uh oh!

github-actions Bot commented May 21, 2026

Uh oh!

amoeba commented May 21, 2026

Uh oh!

amoeba commented May 21, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

amoeba commented Jul 2, 2026

Uh oh!

github-actions Bot commented Jul 2, 2026

Uh oh!

amoeba commented Jul 2, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

amoeba commented Jul 2, 2026

Uh oh!

pitrou commented Jul 2, 2026

Uh oh!

DerThorsten commented Jul 2, 2026

Uh oh!

pitrou commented Jul 2, 2026

Uh oh!

amoeba commented Jul 2, 2026

Uh oh!

github-actions Bot commented Jul 2, 2026

Uh oh!

rok commented Jul 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

amoeba commented May 19, 2026 •

edited

Loading

rok commented Jul 3, 2026 •

edited

Loading