process: add threadCpuUsage #56467

ShogunPanda · 2025-01-04T06:54:43Z

This PR add the threadCpuUsage method to process.
The method works exactly like cpuUsage but it returns thread specific metrics.
This is already implemented (by me :)) in user-land in https://www.npmjs.com/package/thread-cpu-usage.

The PR is currently a draft as I'm gonna use the CI to see which platform don't support this. I'll add documentation updates once this is solved.

nodejs-github-bot · 2025-01-04T06:54:48Z

Review requested:

@nodejs/startup

nodejs-github-bot · 2025-01-04T06:58:10Z

CI: https://ci.nodejs.org/job/node-test-pull-request/64336/

santigimeno · 2025-01-04T08:18:23Z

What about moving the platform dependent code to libuv? Maybe taking libuv/libuv#3120 to the finish line would be a good idea.

anonrig

LGTM except couple of questions and concerns

src/node_process_methods.cc

anonrig · 2025-01-05T20:43:24Z

lib/internal/process/per_thread.js

@@ -148,6 +149,46 @@ function wrapProcessMethods(binding) {
    };
  }

+  const threadCpuValues = new Float64Array(2);


I recommend moving this to C++ side and updating it. A similar implementation exist in node url

Will look it up. Thanks.

I tried to look it up in the code but I couldn't find it.
Do you mind linking a reference to the similar implementation so I can check it out?

src/node_process_methods.cc

anonrig · 2025-01-05T20:48:00Z

lib/internal/process/per_thread.js

+  // Replace the native function with the JS version that calls the native
+  // function.


Why don't we just implement the whole thing in C++? I’m a bit confused about this particular comment.

Well, two reasons:

I'm not totally familiar with C++ :)

I want to keep the most parts we can in JS so that will be easier for a bigger chunk of contributors to chime in.

anonrig · 2025-01-05T20:48:36Z

lib/internal/process/per_thread.js

+    if (prevValue) {
+      if (!previousValueIsValid(prevValue.user)) {
+        validateObject(prevValue, 'prevValue');
+
+        validateNumber(prevValue.user, 'prevValue.user');
+        throw new ERR_INVALID_ARG_VALUE.RangeError('prevValue.user',
+                                                   prevValue.user);
+      }
+
+      if (!previousValueIsValid(prevValue.system)) {
+        validateNumber(prevValue.system, 'prevValue.system');
+        throw new ERR_INVALID_ARG_VALUE.RangeError('prevValue.system',
+                                                   prevValue.system);
+      }
+    }


It seems a lot of this functionality can be removed if we move the implementation to cpp?

See above. :)
Anyway, we would have to perform the validation in the C++ side anyway, isn't it?

ShogunPanda · 2025-01-07T06:50:09Z

What about moving the platform dependent code to libuv? Maybe taking libuv/libuv#3120 to the finish line would be a good idea.

I'm sorry but I have very limited bandwidth lately. Trying to contribute to a codebase I never already did to it's not feasible for me.

Once this lands everyone is absolutely welcome to port this to libuv.

nodejs-github-bot · 2025-01-07T09:31:51Z

CI: https://ci.nodejs.org/job/node-test-pull-request/64382/

nodejs-github-bot · 2025-01-07T14:10:15Z

CI: https://ci.nodejs.org/job/node-test-pull-request/64384/

juanarbol · 2025-01-07T21:01:11Z

What about moving the platform dependent code to libuv? Maybe taking libuv/libuv#3120 to the finish line would be a good idea.

if that PR is orphaned, I can work on that one. Can I @jasnell?

ShogunPanda · 2025-01-08T05:00:32Z

@juanarbol Ok, I'll pause on this until you update me about libuv PR. Thanks for taking care of that!

juanarbol · 2025-01-09T03:54:05Z

@juanarbol Ok, I'll pause on this until you update me about libuv PR. Thanks for taking care of that!

There we go libuv/libuv#4666

ShogunPanda · 2025-01-09T05:09:41Z

Amazing! Keep up posted when this is released in libuv so I can update the dependencies and make this PR much leaner :)

Do you have an approximate ETA?

juanarbol · 2025-01-09T16:40:21Z

Do you have an approximate

I do not. They asked for macOS support, I'm gonna send patches on my spare time, I think I have some slots today and tomorrow after work. But idk when is the next release.

I just gave support to macOS and addressed all the comments.

juanarbol · 2025-01-14T19:56:58Z

@ShogunPanda Paolo, ~~the feature is landed and it is about to be released see libuv/libuv#4667~~

The feature is released!

ShogunPanda · 2025-01-18T07:50:33Z

@juanarbol Thanks for your effort on this.
Since #56616 has landed, I updated this to use the new API.
This is now ready to be reviewed!

ShogunPanda · 2025-02-04T13:45:41Z

@nodejs/build @juanarbol @jasnell I keep changing the worker thread schedule period in the failing test but the thread CPU usage ratio between threads is always very close to 1, while it shouldn't. Can you think of any characteristic of test machines that might explain this?

Most of them are virtual servers? (The implementation of this is hidden from us by most of the providers.)

~~So in this case I guess I can just check I receive numbers and do not care about the real value, isn't it? Do you think it's enough?~~

~~Thanks to the live (a.k.a. in person) suggestion of @mcollina I'm moving this to sequentials, let's hope it fixes it.~~

Nevermind, it hasn't changed anything. I've changed the test to only check we receive positive numbers.

nodejs-github-bot · 2025-02-10T10:09:52Z

CI: https://ci.nodejs.org/job/node-test-pull-request/65115/

jasnell · 2025-02-10T15:04:22Z

There's at least one relevant failure in the latest CI run still.

mcollina · 2025-02-11T11:32:12Z

test/parallel/test-process-threadCpuUsage-worker-threads.js

+// This block can be removed once SmartOS support is fixed in
+// https://github.com/nodejs/node/pull/56467#issuecomment-2628877767
+if (isSunOS) {
+  skip('Operation not supported on SmartOS');


Given it's a public API, I think having a test showing the behavior that happens in SmartOS is sound. Also, don't link to a comment, but open an issue on libuv for this.

mcollina · 2025-02-11T11:32:59Z

lib/internal/process/per_thread.js

+
+  // Replace the native function with the JS version that calls the native
+  // function.
+  function threadCpuUsage(prevValue) {


Can you throw a proper error if this is run on SmartOS?

ShogunPanda requested review from mcollina, anonrig and joyeecheung January 4, 2025 06:54

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. process Issues and PRs related to the process subsystem. labels Jan 4, 2025

ShogunPanda added the request-ci Add this label to start a Jenkins CI on a PR. label Jan 4, 2025

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jan 4, 2025

anonrig reviewed Jan 5, 2025

View reviewed changes

ShogunPanda added the request-ci Add this label to start a Jenkins CI on a PR. label Jan 7, 2025

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jan 7, 2025

ShogunPanda added the request-ci Add this label to start a Jenkins CI on a PR. label Jan 7, 2025

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jan 7, 2025

santigimeno mentioned this pull request Jan 9, 2025

misc: implement uv_getrusage_thread libuv/libuv#4666

Merged

ShogunPanda force-pushed the thread-cpu-usage branch from 1fdd439 to 5b09f78 Compare January 18, 2025 07:48

ShogunPanda marked this pull request as ready for review January 18, 2025 07:50

ShogunPanda force-pushed the thread-cpu-usage branch from d7a6bf1 to 94a4c28 Compare February 5, 2025 08:13

ShogunPanda added the request-ci Add this label to start a Jenkins CI on a PR. label Feb 5, 2025

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Feb 5, 2025

This comment was marked as outdated.

Sign in to view

ShogunPanda force-pushed the thread-cpu-usage branch 2 times, most recently from 585603d to f3786f6 Compare February 8, 2025 08:26

ShogunPanda added the request-ci Add this label to start a Jenkins CI on a PR. label Feb 8, 2025

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Feb 8, 2025

ShogunPanda added the request-ci Add this label to start a Jenkins CI on a PR. label Feb 8, 2025

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Feb 8, 2025

ShogunPanda added the request-ci Add this label to start a Jenkins CI on a PR. label Feb 8, 2025

jasnell removed the request-ci Add this label to start a Jenkins CI on a PR. label Feb 8, 2025

This comment was marked as outdated.

Sign in to view

jasnell approved these changes Feb 8, 2025

View reviewed changes

ShogunPanda added the request-ci Add this label to start a Jenkins CI on a PR. label Feb 10, 2025

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Feb 10, 2025

This comment was marked as outdated.

Sign in to view

ShogunPanda force-pushed the thread-cpu-usage branch 2 times, most recently from 1d114aa to 620c4d6 Compare February 10, 2025 08:50

ShogunPanda added the request-ci Add this label to start a Jenkins CI on a PR. label Feb 10, 2025

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Feb 10, 2025

This comment was marked as outdated.

Sign in to view

process: add threadCpuUsage

7b88cb9

ShogunPanda force-pushed the thread-cpu-usage branch from 620c4d6 to 7b88cb9 Compare February 10, 2025 09:53

ShogunPanda added the request-ci Add this label to start a Jenkins CI on a PR. label Feb 10, 2025

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Feb 10, 2025

mcollina reviewed Feb 11, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

process: add threadCpuUsage #56467

process: add threadCpuUsage #56467

ShogunPanda commented Jan 4, 2025

nodejs-github-bot commented Jan 4, 2025

nodejs-github-bot commented Jan 4, 2025

santigimeno commented Jan 4, 2025

anonrig left a comment

anonrig Jan 5, 2025

ShogunPanda Jan 7, 2025

ShogunPanda Jan 7, 2025

anonrig Jan 5, 2025

ShogunPanda Jan 7, 2025

anonrig Jan 5, 2025

ShogunPanda Jan 7, 2025

ShogunPanda commented Jan 7, 2025

nodejs-github-bot commented Jan 7, 2025

nodejs-github-bot commented Jan 7, 2025

juanarbol commented Jan 7, 2025

ShogunPanda commented Jan 8, 2025

juanarbol commented Jan 9, 2025

ShogunPanda commented Jan 9, 2025

juanarbol commented Jan 9, 2025 •

edited

Loading

juanarbol commented Jan 14, 2025 •

edited

Loading

ShogunPanda commented Jan 18, 2025

ShogunPanda commented Feb 4, 2025 •

edited

Loading

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

nodejs-github-bot commented Feb 10, 2025

jasnell commented Feb 10, 2025

mcollina Feb 11, 2025

mcollina Feb 11, 2025

		// Replace the native function with the JS version that calls the native
		// function.

process: add threadCpuUsage #56467

Are you sure you want to change the base?

process: add threadCpuUsage #56467

Conversation

ShogunPanda commented Jan 4, 2025

nodejs-github-bot commented Jan 4, 2025

nodejs-github-bot commented Jan 4, 2025

santigimeno commented Jan 4, 2025

anonrig left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ShogunPanda commented Jan 7, 2025

nodejs-github-bot commented Jan 7, 2025

nodejs-github-bot commented Jan 7, 2025

juanarbol commented Jan 7, 2025

ShogunPanda commented Jan 8, 2025

juanarbol commented Jan 9, 2025

ShogunPanda commented Jan 9, 2025

juanarbol commented Jan 9, 2025 • edited Loading

juanarbol commented Jan 14, 2025 • edited Loading

ShogunPanda commented Jan 18, 2025

ShogunPanda commented Feb 4, 2025 • edited Loading

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

nodejs-github-bot commented Feb 10, 2025

jasnell commented Feb 10, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

juanarbol commented Jan 9, 2025 •

edited

Loading

juanarbol commented Jan 14, 2025 •

edited

Loading

ShogunPanda commented Feb 4, 2025 •

edited

Loading