Need clarification on disabling PARALLEL_QUERY_RUNNING at a runtime

[mehov]

Preliminary Checks

• This issue is not a duplicate. Before opening a new issue, please search existing issues: https://github.com/gatsbyjs/gatsby/issues
• This issue is not a question, feature request, RFC, or anything other than a bug report. Please post those things in GitHub Discussions: https://github.com/gatsbyjs/gatsby/discussions

Summary

Hi everyone,

Keeping it short.

What I want

Disable PARALLEL_QUERY_RUNNING because I'm getting these errors:

• ERROR #85928
• /project/node_modules/yoga-layout-prebuilt/yoga-layout/build/Release/nbind.js:53 & Error: write EPIPE

What I tried:

• PARALLEL_QUERY_RUNNING=false gatsby build

• GATSBY_PARALLEL_QUERY_RUNNING=false gatsby build

• adding

  module.exports = {
    flags: {
      PARALLEL_QUERY_RUNNING: false
    }
  }
  

  to gatsby-config.js results in:

  Some features you configured with flags are used natively now.
  Those flags no longer have any effect and you can remove them from config:

  • PARALLEL_QUERY_RUNNING · EXPERIMENTAL · (Umbrella Issue (https://gatsby.dev/pqr-feedback)) · Parallelize running page queries in order to better saturate all available cores. Improves time it takes to run queries during gatsby build.

Gatsby Info

  System:
    OS: Linux 2.6 Debian GNU/Linux 9 (stretch) 9 (stretch)
    CPU: (2) x64 Intel(R) Xeon(R) CPU E3-1240 V2 @ 3.40GHz
    Shell: 4.4.12 - /bin/bash
  Binaries:
    Node: 14.18.3 - /usr/bin/node
    npm: 6.14.15 - /usr/bin/npm
  Languages:
    Python: 2.7.13 - /usr/bin/python
  npmPackages:
    gatsby: ^4.0.0 => 4.5.0
    gatsby-plugin-image: ^2.0.0 => 2.5.0
    gatsby-plugin-manifest: ^4.0.0 => 4.5.0
    gatsby-plugin-matomo: ^0.11.0 => 0.11.0
    gatsby-plugin-offline: ^5.0.0 => 5.5.0
    gatsby-plugin-react-helmet: ^5.0.0 => 5.5.0
    gatsby-plugin-sharp: ^4.0.0 => 4.5.0
    gatsby-plugin-sitemap: ^5.3.0 => 5.5.0
    gatsby-source-filesystem: ^4.0.0 => 4.5.0
    gatsby-source-wordpress: ^6.0.0 => 6.5.0
    gatsby-transformer-sharp: ^4.0.0 => 4.5.0
  npmGlobalPackages:
    gatsby-cli: 4.5.0

Steps to Resolve this Issue

Please help

[mehov]

Searched for 85928, found GATSBY_PARALLEL_QUERY_CHUNK_SIZE and Improvements to Parallel Query Running, tried GATSBY_PARALLEL_QUERY_CHUNK_SIZE=10 gatsby build --verbose, didn't help:

Building production JavaScript and CSS bundles
lscpu: failed to determine number of CPUs: /sys/devices/system/cpu/possible: No such file or directory
success Building production JavaScript and CSS bundles - 3.761s
success Building HTML renderer - 2.622s
success Execute page configs - 0.053s
success Caching Webpack compilations - 0.001s
success run queries in workers - 0.023s - 0/42 1861.48/s
success Merge worker state - 0.001s

ERROR #85928

An error occurred during parallel query running.
Go here for troubleshooting tips: https://gatsby.dev/pqr-feedback

Error: Worker exited before finishing task

• index.js:193
  [app]/[gatsby-worker]/dist/index.js:193:31

[mehov]

According to this comment by @LekoArts, an 859 error could actually mean there's something wrong with my GraphQL, so the cause for the error could be on the Wordpress side of things?

Our error map has a loose format, e.g. all GraphQL errors are prefixed with 859 (…)

[mehov]

According to this comment,

This issue is resolved after updating a few other plugins (…)

so I ran npm update, and while some plugins did upgrade, it still didn't help. Here's what changed in my package.json:

   "dependencies": {
     "acorn": "^8.6.0",
-    "gatsby": "^4.0.0",
-    "gatsby-plugin-image": "^2.0.0",
-    "gatsby-plugin-manifest": "^4.0.0",
+    "gatsby": "^4.5.2",
+    "gatsby-plugin-image": "^2.5.1",
+    "gatsby-plugin-manifest": "^4.5.1",
     "gatsby-plugin-matomo": "^0.11.0",
-    "gatsby-plugin-offline": "^5.0.0",
+    "gatsby-plugin-offline": "^5.5.1",
     "gatsby-plugin-react-helmet": "^5.0.0",
-    "gatsby-plugin-sharp": "^4.0.0",
+    "gatsby-plugin-sharp": "^4.5.1",
     "gatsby-plugin-sitemap": "^5.3.0",
-    "gatsby-source-filesystem": "^4.0.0",
-    "gatsby-source-wordpress": "^6.0.0",
+    "gatsby-source-filesystem": "^4.5.1",
+    "gatsby-source-wordpress": "^6.5.1",
     "gatsby-transformer-sharp": "^4.0.0",
     "html-react-parser": "^1.1.1",
     "lodash": "^4.17.20",

[mehov]

so the cause for the error could be on the Wordpress side of things?

The issue I'm reporting happens on my VPS that runs Debian. I have a copy of this repo hosted on GatsbyJS.com and it feeds into the same GraphQL Wordpress endpoint. I have triggered a manual build on GatsbyJS.com and it built just fine. So this must be something else and not on the Wordpress side.

[LekoArts]

You can't disable PQR in v4 -- your VPS might not have enough memory and thus errors out. We're still working on improving the memory footprint for the workers that are used in PQR.

[LekoArts]

Reducing the GATSBY_CPU_COUNT is the best workaround at the moment as it forces it to not spawn any additional workers

[panzacoder]

@LekoArts Reducing GATSBY_CPU_COUNT has no effect for me on v4.6 or v4.5. Currently this issue is affecting 100% of my builds.

Adjustments I have tried, some of which seemed to work but ultimately proved to be false postives:

• Rolling back to v4.5 (I had just upgraded from 2.x to 4.6 when this problem started)
• Pruning some of the larger images in contentful
• Setting GATSBY_CPU_COUNT to 3, 2, and 1
• Turning off PARALLEL_SOURCING
• Removing gatsby-remark-images-contentful (I thought I had tracked this issue to the step when it downloads the Base64 image)
• Updating Node in my build settings from v14 to v16
• Adding, removing, and changing version for the netlify-plugin-gatsby
• Updating gatsby-plugin-netlify

Nothing has worked. It's causing real problems, and at this point my options are to upgrade to the high-performance builds at Netlify, or to attempt to revert to gatsby v3.x, but regressing plugins is a nightmare.

Any help or updates on this very serious issue for me would be much appreciated.

[SergiOca87]

This happens to us, for a colleague locally, for me locally and on Gatsby Cloud, we need to disable it. It is completely halting the work on a production site.

[gczudowski]

HI, I have the same issue, any news on this?

[RazvanBejamas]

After upgrading to v4 for a website with 11.000+ pages, I have the same issue. I enabled PARALLEL_QUERY_RUNNING for testing in v3 and it fail at the same point as after upgrading to v4. It will be nice to be able to disable this feature in v4.

[milde-marketing]

I have the same problem (upgrading from v2 to v4). The build is running locally, no issues. On my VPS it's probably like the previous speakers. Same problem, GATSBY_CPU_COUNT=1 in my production env fixes it for now, but it's very very slow. It would be great if you can fix this with sth like a PQR disable function or less memory usage. Thank you!

[robwilkerson]

Any chance there are any updates on this? I see lots of problem statements, but haven't yet found a working solution.

[gorohoroh]

No chance, I'm afraid. The Gatsby team has historically sucked at both providing adequate troubleshooting output during build and at responding to troubleshooting requests here on GitHub. Following the Netlify merger, there's basically no one left to suck at these things because the majority of the team has been laid off.

I'm inclined to say something about karma, but I'd better spend effort on evaluating migration to Astro.

[panzacoder]

FWIW Netlify Connect is the spiritual successor to Gatsby's Source layer. If that part of the ecosystem was valuable to you, I would suggest trying that out and using your components in a Next or Remix app.

There is a lot of hype for Remix, but personally as a former Gatsby dev I have been loving the Next App Dir and RSC. If I get the chance to another build for a large marketing team again, I think I would still try to use Netlify Connect for the ease of integration with my content layer.