11GB+ and 15min+ on "extract queries from components" step of build. Why?

[miloofcroton]

Hello all,

I'm inheriting an older Gatsby project, and I've faced considerable issues with upgrading the app from Gatsby 2 to Gatsby 3 and 4. Currently on 4.25.6.

I finally have the site building with Gatsby 4, but ONLY after I gave the process 16GB of RAM. I didn't pay close attention, but it was using at least 11-12GB at peak.

On what, you might ask? It was primarily "extract queries from components". It logged two separate times in the console, both at close to 20 minutes for each log.

I'm trying to understand what is causing this memory issue and extremely long build process?

It appears to be compiling all instances of useStaticQuery, but I don't have a ton of insight yet. I gather it renders every page on the site — would this be independently rendering every react component under src, or is it only rendering src/pages as an entry point directory? I assume all of src, since you can apparently use that hook throughout that dir.

I'm wondering if it would help to move all static queries to their page components and pass down data as props?

Do you know if it deeply or shallowly renders each of this react components during query extraction?

I did dig into the library files, starting with query-compiler.js as well as file-parser.js, and I found async file reading, which is a bit of a concern.

Any tips? I have googled this build step, and it seems a lot of people have out of memory issues with this, so it must be well-known in the community.

I am willing to attempt to upgrade to Gatsby 5 if that means better memory usage.

I should add that it previously was not a fast build on Gatsby 2, but it couldn't have been more than 5-7 minutes.

I'm also really concerned about this layout component file that has 6 context providers plus 2 calls to browser-specific APIs (history and window.addeventlistener). I could potentially refactor those things out of the layout component and put them into a normal component tree? But because I'm not sure how everything is rendered during the build (deep vs shallow, just pages with useStaticQuery vs all, etc) that I'm not sure if this would fix anything.

Plz help.

[miloofcroton]

Problem solved. The type file being auto-generated was insanely big and being required in many components, leading to insane amounts of file loading, potentially duplicated several times if they were used multiple times on pages. This was breaking on one of the graphql util functions somewhere in file parser, "findGraphQLTags".