Darwin: scan thread information #546

cgzones · 2021-02-28T18:41:23Z

Closes: #542

/cc @robaho @Karsten1987 @fasterit

natoscott · 2021-03-01T02:41:39Z

@cgzones looks good to me. Might be worth setting 'isThread = false;' in DarwinProcess_new instead of relying on the calloc there, for code clarity. If that's not preferred and calloc is OK - which is fine by me - there's some other initializations-to-zero in that function that could be removed.

Also I wonder about the taskAccess field (not from your new code, but related to threads) - it seems like this field might cause incorrect behaviour if htop samples a process before it creates any threads - if it only creates threads later in its life, those threads would never be counted because this flag was set first time through. If thats the case, I think it could be safely removed - it appears to be an optimization only, AFAICS.

Karsten1987

I honestly don't really see a difference with this change.

I've added a quick screenshot of my local output on my macbook, where the executable a.out contains two concurrently running pthreads. However, these are not visualized in any way.

For comparison, the same application built for Linux clearly shows the two running threads (highlighted in green):

Karsten1987 · 2021-03-01T21:47:07Z

darwin/DarwinProcess.c

-      mach_msg_type_number_t thread_info_count = THREAD_BASIC_INFO_COUNT;
-      ret = thread_info(thread_list[i], THREAD_BASIC_INFO, (thread_info_t)thinfo, &thread_info_count);
+      mach_msg_type_number_t thread_info_count = THREAD_EXTENDED_INFO_COUNT;
+      ret = thread_info(thread_list[i], THREAD_EXTENDED_INFO_COUNT, (thread_info_t)thinfo, &thread_info_count);


are you sure that the thinfo struct is passed by copy and not by pointer-to? I've seen a couple of SO posts passing it in via (thread_info_t) &thinfo: https://stackoverflow.com/a/33414855/4583130

the same actually applies to the similar logic above for obtaining the TASK_BASIC_INFO.

BenBE · 2021-03-01T22:08:27Z

Just a small tip: Press Esc once to hide the process list cursor. Also Tree Mode might be of help to group the threads optically below their parent.

darwin/DarwinProcess.h

darwin/DarwinProcess.c

darwin/DarwinProcess.h

BenBE · 2021-05-24T11:48:07Z

Please check if this also fixes #622.

Karrq · 2023-04-08T15:44:19Z

I checked out this lcoally and built it but I still don't see the threads, even running the tool with sudo.

On MBP M2 Ventura 13.3

BenBE · 2023-04-08T18:16:08Z

Have you checked that your configuration has display of userland threads activated.

Karrq · 2023-04-09T09:28:15Z

Yep I had unchecked "Hide userland process threads" and also tried toggling it on and off with H just to make sure and I don't see any change in my output, and I also tried tree/list mode...
I was trying to look at the threads of a specific process that I'm interested in as Activity Monitor only reports the number of threads (8), so I'm sure there are threads for that process.

(btw v3.2.2 also doesn't show threads per process, but I can see the total number of user threads under tasks)

BenBE · 2023-04-09T13:35:24Z

Okay, strange. I can confirm this is somewhat strange behaviour. @cgzones Can you check your patch again?

cgzones · 2023-04-12T17:19:26Z

Recent versions of MacOS limit the access to foreign threads via thread_info(). I am not aware of an alternative API.
It might work though with a signed binary running as root (not tested).

hashok · 2023-08-30T10:30:55Z

I've compiled htop from this branch on MacBook Pro Max M2 running macOS Ventura 13.5.1 (all updates installed as of 30-Aug-2023) and I do see user land threads 🎉 , but only when running sudo htop. I have not signed the htop binary indeed. So it's definitely an improvement over the current htop version 3.2.2 from Homebrew.

Here are worker threads of the perf_test tool shows. Also note alignment issues of | in the tree view. But that's a different issue indeed.

emadpres · 2023-08-30T13:28:54Z

I also just compiled cgzones:darwin_threads on M2 Max (McaOS 13.5.1), and I can confirm the threads are shown (only with sudo).

Not sure how relevant, the followings are broken:

refreshing
Tree structure (see |s)
scrolling

here is a side-by-side comparison of htop v3.2.2 with the compiled branch:

polluks · 2023-08-30T21:02:01Z

Works on macOS 14.0
@emadpres Your PID is really high, too high for the tree.

emadpres · 2023-08-31T11:34:25Z

Works on macOS 14.0 @emadpres Your PID is really high, too high for the tree.

What do you mean it's too high? That's what I get.

BenBE · 2023-08-31T11:37:37Z

@cgzones Can you rebase? What'S missing to push this along?

polluks · 2023-08-31T21:23:27Z

@emadpres I was talking about the width: Your PID have 7 digits but htop is fixed for 5 digits.
This seems to be a bug. However your system should not use 3 million processes in 4 days...

Inspired by: hishamhm/htop#848 Closes: htop-dev#542

cgzones · 2023-09-13T15:45:46Z

What's missing to push this along?

Testing: check all threads are displayed, no memory is leaked, tree glitches.

(Only compile tested the latest rebase.)

BenBE

LGTM. No obvious memleaks.

Minor nitpick: Neither identifer_info_count nor extended_info_count are used or checked after their assignment through the call to thread_info, but access to at least the first item of the returned arrays is assumed.

cgzones · 2023-09-13T17:51:48Z

LGTM. No obvious memleaks.

Minor nitpick: Neither identifer_info_count nor extended_info_count are used or checked after their assignment through the call to thread_info, but access to at least the first item of the returned arrays is assumed.

The output parameters identifer_info and extended_info are not arrays, see https://web.mit.edu/darwin/src/modules/xnu/osfmk/man/thread_info.html for API details.

BenBE · 2023-09-13T19:05:20Z

LGTM. No obvious memleaks.
Minor nitpick: Neither identifer_info_count nor extended_info_count are used or checked after their assignment through the call to thread_info, but access to at least the first item of the returned arrays is assumed.

The output parameters identifer_info and extended_info are not arrays, see https://web.mit.edu/darwin/src/modules/xnu/osfmk/man/thread_info.html for API details.

Ah, I see. So basically buffer sizes. In that case, at least checking for "a whole item was returned" should suffice. But as mentioned, that is somewhat nitpicky.

BenBE · 2023-10-08T22:07:35Z

@cgzones Can you have a look at the PID auto-sizing issue? Likely missing the width update call for that column. Bugfix for that should get its own PR.

@fasterit Can you take a look at this PR, so we can push it along?

dundee · 2024-04-19T15:36:29Z

Any update on this? Would be great to see threads on Mac!

fasterit · 2024-04-19T16:09:18Z

@dundee: Did you test the cgzones:darwin_threads branch?

dundee · 2024-04-20T21:32:38Z

@fasterit Yes, it works fine for me (using sudo).

findepi · 2024-07-11T15:20:26Z

@cgzones @BenBE thanks for this PR!
Small question, which version this is/will be available in?
latest version from brew is 3.3.0 and doesn't show threads (even with sido). Current main branch built from source shows threads on my Mac (when run with sudo). The from-source build also reports version as 3.3.0.

BenBE · 2024-07-11T16:28:42Z

If your from-source build reports 3.3.0 you should re-run autogen.sh.

That aside: from the commit mentioned in the merge commit you can determine exactly when this landed, but I'd guess it's highly likely this was after 3.3.0 was released; thus this is probably not included in any release yet. Have a look at the milestone of this PR to check for which release it is planned.

findepi · 2024-07-11T16:55:24Z

@BenBE apologies. i checked version wrong. my from-source build is 3.4.0-dev-3.3.0-144-gdb73229.

Have a look at the milestone of this PR to check for which release it is planned.

oh, i didn't notice that, but that's so nice.
looking forward to 3.4.0 release then.
thanks!

cgzones mentioned this pull request Feb 28, 2021

No threads on OSX #542

Closed

Karsten1987 reviewed Mar 1, 2021

View reviewed changes

BenBE added the MacOS 🍏 MacOS / Darwin related issues label Mar 23, 2021

BenBE requested changes May 23, 2021

View reviewed changes

darwin/DarwinProcess.h Outdated Show resolved Hide resolved

darwin/DarwinProcess.c Outdated Show resolved Hide resolved

BenBE added the needs-rebase Pull request needs to be rebased and conflicts to be resolved label May 23, 2021

cgzones force-pushed the darwin_threads branch from 6202da5 to 97da8f4 Compare May 23, 2021 13:31

cgzones removed the needs-rebase Pull request needs to be rebased and conflicts to be resolved label May 23, 2021

cgzones force-pushed the darwin_threads branch 3 times, most recently from fcc37d8 to 92814d7 Compare May 23, 2021 13:40

BenBE reviewed May 23, 2021

View reviewed changes

darwin/DarwinProcess.c Outdated Show resolved Hide resolved

darwin/DarwinProcess.h Outdated Show resolved Hide resolved

BenBE added the enhancement Extension or improvement to existing feature label May 24, 2021

BenBE mentioned this pull request Dec 10, 2021

Threads info not available on Macos11.4 with htop3.1.2 #886

Closed

cgzones force-pushed the darwin_threads branch 3 times, most recently from 2dc145c to 4059f8e Compare April 12, 2023 17:12

darwin: avoid double division by zero on startup

1b68c1a

cgzones force-pushed the darwin_threads branch 3 times, most recently from d9f9237 to d62f335 Compare September 13, 2023 15:38

[WIP] Darwin: scan thread information

357bbe6

Inspired by: hishamhm/htop#848 Closes: htop-dev#542

cgzones force-pushed the darwin_threads branch from d62f335 to 357bbe6 Compare September 13, 2023 15:43

BenBE marked this pull request as ready for review September 13, 2023 17:02

BenBE approved these changes Sep 13, 2023

View reviewed changes

BenBE added this to the 3.3.0 milestone Oct 8, 2023

BenBE modified the milestones: 3.3.0, 3.4.0 Dec 26, 2023

BenBE merged commit 423d183 into htop-dev:main Apr 20, 2024

cgzones deleted the darwin_threads branch April 26, 2024 17:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Darwin: scan thread information #546

Darwin: scan thread information #546

cgzones commented Feb 28, 2021

natoscott commented Mar 1, 2021

Karsten1987 left a comment

Karsten1987 Mar 1, 2021

BenBE commented Mar 1, 2021

BenBE commented May 24, 2021

Karrq commented Apr 8, 2023

BenBE commented Apr 8, 2023

Karrq commented Apr 9, 2023

BenBE commented Apr 9, 2023

cgzones commented Apr 12, 2023 •

edited

Loading

hashok commented Aug 30, 2023 •

edited

Loading

emadpres commented Aug 30, 2023

polluks commented Aug 30, 2023

emadpres commented Aug 31, 2023

BenBE commented Aug 31, 2023

polluks commented Aug 31, 2023

cgzones commented Sep 13, 2023

BenBE left a comment

cgzones commented Sep 13, 2023

BenBE commented Sep 13, 2023

BenBE commented Oct 8, 2023

dundee commented Apr 19, 2024

fasterit commented Apr 19, 2024

dundee commented Apr 20, 2024

findepi commented Jul 11, 2024 •

edited

Loading

BenBE commented Jul 11, 2024

findepi commented Jul 11, 2024

Darwin: scan thread information #546

Darwin: scan thread information #546

Conversation

cgzones commented Feb 28, 2021

natoscott commented Mar 1, 2021

Karsten1987 left a comment

Choose a reason for hiding this comment

Karsten1987 Mar 1, 2021

Choose a reason for hiding this comment

BenBE commented Mar 1, 2021

BenBE commented May 24, 2021

Karrq commented Apr 8, 2023

BenBE commented Apr 8, 2023

Karrq commented Apr 9, 2023

BenBE commented Apr 9, 2023

cgzones commented Apr 12, 2023 • edited Loading

hashok commented Aug 30, 2023 • edited Loading

emadpres commented Aug 30, 2023

polluks commented Aug 30, 2023

emadpres commented Aug 31, 2023

BenBE commented Aug 31, 2023

polluks commented Aug 31, 2023

cgzones commented Sep 13, 2023

BenBE left a comment

Choose a reason for hiding this comment

cgzones commented Sep 13, 2023

BenBE commented Sep 13, 2023

BenBE commented Oct 8, 2023

dundee commented Apr 19, 2024

fasterit commented Apr 19, 2024

dundee commented Apr 20, 2024

findepi commented Jul 11, 2024 • edited Loading

BenBE commented Jul 11, 2024

findepi commented Jul 11, 2024

cgzones commented Apr 12, 2023 •

edited

Loading

hashok commented Aug 30, 2023 •

edited

Loading

findepi commented Jul 11, 2024 •

edited

Loading