Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: refactor the generateBlogData function (again) (#7607) #7618

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

dario-piotrowicz
Copy link
Member

@dario-piotrowicz dario-piotrowicz commented Apr 3, 2025

Description

Reintroducing the refactoring I added in #7607 that actually broke production ( 🙈 ) and I had to quickly revert in #7617

What went wrong / what I changed from my previous PR

Basically with my previous code the Next.js compiler would move code around breaking things, this is the code it generated for my previous iteration of the file:
Screenshot at 2025-04-03 23-28-17

The problem being that f (which is the minified version of blogCategories) gets spread in the result before the await Promise.all that actually side-effectfully populates it

In this PR I am then also moving blogCategories inside generateBlogData , this is arguably cleaner than the previous iteration anyways (in my opinion) and it seems to help the Next.js compiler not to move things around, no longer introducing the issue

Screenshot at 2025-04-03 23-34-10

(as you can see now the await Promise.all is correctly run before the return statement)

Validation

I've manually tested this both in next dev and next build + next start (both this time! 🙈)

Related Issues

Check List

  • I have read the Contributing Guidelines and made commit messages that follow the guideline.
  • I have run npm run format to ensure the code follows the style guide.
  • I have run npm run test to check if all tests are passing.
  • I have run npx turbo build to check if the website builds without errors.
  • I've covered new added functionality with unit tests if necessary.

@Copilot Copilot bot review requested due to automatic review settings April 3, 2025 22:38
@dario-piotrowicz dario-piotrowicz requested a review from a team as a code owner April 3, 2025 22:38
Copy link

vercel bot commented Apr 3, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated (UTC)
nodejs-org ✅ Ready (Inspect) Visit Preview Apr 5, 2025 11:43pm

Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 1 out of 1 changed files in this pull request and generated no comments.

Comments suppressed due to low confidence (2)

apps/site/next-data/generators/blogData.mjs:70

  • [nitpick] Consider renaming 'rawFrontmatter' to 'frontmatterContent' to more clearly indicate that it stores the accumulated Markdown frontmatter text.
let rawFrontmatter = '';

apps/site/next-data/generators/blogData.mjs:60

  • Consider adding unit tests to verify that the concurrent file processing logic (including frontmatter extraction and category aggregation) handles typical and edge-case scenarios correctly.
const posts = await Promise.all(

Copy link
Contributor

github-actions bot commented Apr 3, 2025

Lighthouse Results

URL Performance Accessibility Best Practices SEO Report
/en 🔴 66 🟢 100 🟢 100 🟢 91 🔗
/en/about 🟢 100 🟢 100 🟢 100 🟢 91 🔗
/en/about/previous-releases 🟢 99 🟢 100 🟢 100 🟢 92 🔗
/en/download 🟢 97 🟢 100 🟢 100 🟢 91 🔗
/en/blog 🟢 100 🟢 100 🟢 96 🟢 92 🔗

@bmuenzenmeyer
Copy link
Collaborator

i appreciate the extra effort in testing this, but i'd really like if we could cover this scenario with a test - either unit or lighthouse routes and #7395

Copy link
Contributor

github-actions bot commented Apr 3, 2025

Unit Test Coverage Report

Title Lines Statements Branches Functions
@node-core/ui-components Coverage: 95%
95.83% (161/168) 77.86% (102/131) 88.57% (31/35)
@nodejs/website Coverage: 88%
85.62% (530/619) 76.68% (171/223) 87.69% (114/130)
Title Tests Skipped Failures Errors Time
@node-core/ui-components 24 0 💤 0 ❌ 0 🔥 5.228s ⏱️
@nodejs/website 161 0 💤 0 ❌ 0 🔥 6.653s ⏱️

@dario-piotrowicz
Copy link
Member Author

i appreciate the extra effort in testing this, but i'd really like if we could cover this scenario with a test - either unit or lighthouse routes and #7395

yeah I totally agree with that 👍

I can of course add some unit tests if you want 🙂

However they would have not caught the regression introduced by my changes anyways since those were due to the Next.js build output 😕 , a full e2e test would be the only thing that would have prevented that

@bmuenzenmeyer
Copy link
Collaborator

a full e2e test would be the only thing that would have prevented that

AFAIK thats what one way #7395 could work - run playwright tests against the vercel preview

@ovflowd
Copy link
Member

ovflowd commented Apr 3, 2025

I'm also sorry for not double checking these myself 🤦 -- I assumed it was working and did not even verify the preview. Because the code made sense.

@@ -39,12 +34,6 @@ const getFrontMatter = (filename, source) => {
// all = (all blog posts), publish year and the actual blog category
const categories = [category, `year-${publishYear}`, 'all'];

// we add the year to the categories set
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why we removing these categories? 👀

Copy link
Member Author

@dario-piotrowicz dario-piotrowicz Apr 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nono, these are still being added anyways to the blogCategories array, I'm just doing it from the categories array here: https://github.com/nodejs/nodejs.org/pull/7618/files/7004629e8ff0dfcdc7a57109d5132fe34ef301c3#diff-5ddcf76b865830c001627fef7abd06107b3eafeb210d21737305db2416a279beR95-R98

@dario-piotrowicz
Copy link
Member Author

dario-piotrowicz commented Apr 3, 2025

I'm also sorry for not double checking these myself 🤦 -- I assumed it was working and did not even verify the preview.

No problem, it also worked on the first page of the blog... so unless you started clicking around it kinda seemed like everything was ok even in the preview 😓

Because the code made sense.

The code sure made sense! it indeed worked perfectly with next dev 😭

@dario-piotrowicz
Copy link
Member Author

a full e2e test would be the only thing that would have prevented that

AFAIK thats what one way #7395 could work - run playwright tests against the vercel preview

I'm a big fan of testing I'd love to help out with that 🙂, anyways it sounds like a bit of a bad timing to add playwright tests right now given a potential migration onto Cloudflare 😓 (#7383)

(or the perfect timing, depending on how you look at it 😅)

@dario-piotrowicz
Copy link
Member Author

i appreciate the extra effort in testing this, but i'd really like if we could cover this scenario with a test - either unit or lighthouse routes and #7395

Hey @bmuenzenmeyer 🙂

I've added some unit tests for the generateBlogData function, please have a look and let me know what you think 🙂

Copy link
Member

@avivkeller avivkeller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't mean to approve, these are just comments

@ovflowd ovflowd added the github_actions:pull-request Trigger Pull Request Checks label Apr 10, 2025
@ovflowd ovflowd enabled auto-merge April 10, 2025 13:27
@github-actions github-actions bot removed the github_actions:pull-request Trigger Pull Request Checks label Apr 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants