[dotnet-trace] Add collect-linux verb

[mdh1418]

Implements dotnet/docs#47894

Following the addition of emitting native runtime and custom EventSource events as user_events through dotnet/runtime#115265 and the public release of https://github.com/microsoft/one-collect which supports collecting both .NET user_events and Linux perf events into a single .nettrace file, dotnet-trace will support a new verb, collect-linux, that wraps around record-trace.

This PR does the following:

• Adds collect-linux verb and serializes a subset of dotnet-trace collect options in addition to a collect-linux specific --perf-events option into record-trace args. (see [Diagnostics][dotnet-trace] Add collect-linux verb docs#47894 for overarching details)
• Adds record-trace dynamic library to dotnet-trace
• Updates existing profiles (cpu-sampling -> dotnet-common + dotnet-sampled-thread-time) and adds collect-linux specific profiles
• Updates list-profiles verb with revamped profiles + multiline description formatting
• Refactors EventPipeProvider composition logic (MergeProfileAndProviders + ToProviders -> ComputeProviderConfig) and rename Extensions.cs -> ProviderUtils.cs
• Revamp EventPipeProvider composition tests (ProviderParsing.cs -> ProviderCompositionTests.cs)
• Various cleanup: Update CLREventKeywords + Update logging + refactor collect logic + expand dotnet-trace common options

Testing

dotnet-trace collect-linux

On Linux

collect-linux

collect-linux --help

$ ./dotnet-trace collect-linux -h
Description:
  Collects diagnostic traces using perf_events, a Linux OS technology. collect-linux requires admin privileges to capture kernel- and user-mode events, and by default, captures events from all processes. This Linux-only command includes the same .NET
  events as dotnet-trace collect, and it uses the kernel’s user_events mechanism to emit .NET events as perf events, enabling unification of user-space .NET events with kernel-space system events.

Usage:
  dotnet-trace collect-linux [options]

Options:
  --providers       A comma delimited list of EventPipe providers to be enabled. This is in the form 'Provider[,Provider]',where Provider is in the form: 'KnownProviderName[:[Flags][:[Level][:[KeyValueArgs]]]]', and KeyValueArgs is in the form:
                    '[key1=value1][;key2=value2]'.  Values in KeyValueArgs that contain ';' or '=' characters need to be surrounded by '"', e.g., FilterAndPayloadSpecs="MyProvider/MyEvent:-Prop1=Prop1;Prop2=Prop2.A.B;".  Depending on your shell, you may
                    need to escape the '"' characters and/or surround the entire provider specification in quotes, e.g., --providers 'KnownProviderName:0x1:1:FilterSpec=\"KnownProviderName/EventName:-Prop1=Prop1;Prop2=Prop2.A.B;\"'. These providers are in
                    addition to any providers implied by the --profile argument. If there is any discrepancy for a particular provider, the configuration here takes precedence over the implicit configuration from the profile.  See documentation for
                    examples.
  --clreventlevel   Verbosity of CLR events to be emitted.
  --clrevents       List of CLR runtime events to emit.
  --perf-events     Comma-separated list of perf events (e.g. syscalls:sys_enter_execve,sched:sched_switch).
  --profile         A named, pre-defined set of provider configurations for common tracing scenarios. You can specify multiple profiles as a comma-separated list. When multiple profiles are specified, the providers and settings are combined (union), and
                    duplicates are ignored.
  -o, --output      The output path for the collected trace data. If not specified it defaults to '<appname>_<yyyyMMdd>_<HHmmss>.nettrace', e.g., 'myapp_20210315_111514.nettrace'. [default: default]
  --duration        When specified, will trace for the given timespan and then automatically stop the trace. Provided in the form of dd:hh:mm:ss.
  -n, --name        The name of the process to collect the trace.
  -p, --process-id  The process id to collect the trace.
  -?, -h, --help    Show help and usage information

`collect-linux` without elevated privileges

$ ./dotnet-trace collect-linux
No providers, profiles, ClrEvents, or PerfEvents were specified, defaulting to trace profiles 'dotnet-common' + 'cpu-sampling'.

Provider Name                           Keywords            Level               Enabled By
Microsoft-Windows-DotNETRuntime         0x000000100003801D  Informational(4)    --profile

Linux Perf Events                                                               Enabled By
cpu-sampling                                                                    --profile

Output File    : /home/mihw/repo/diagnostics/trace_20251008_181812.nettrace

Error: Tracefs is not accessible: Permission denied (os error 13)

`collect-linux` with elevated privileges

$ sudo ./dotnet-trace collect-linux
No providers, profiles, ClrEvents, or PerfEvents were specified, defaulting to trace profiles 'dotnet-common' + 'cpu-sampling'.

Provider Name                           Keywords            Level               Enabled By
Microsoft-Windows-DotNETRuntime         0x000000100003801D  Informational(4)    --profile

Linux Perf Events                                                               Enabled By
cpu-sampling                                                                    --profile

Output File    : /home/mihw/repo/diagnostics/trace_20251008_181939.nettrace

[00:00:00:03]   Recording trace.
Press <Enter> or <Ctrl-C> to exit...

Recording stopped.
Resolving symbols.
Finished recording trace.
Trace written to /home/mihw/repo/diagnostics/trace_20251008_181939.nettrace

On Windows (and I presume other non-Linux OS):

`collect-linux`

.\artifacts\bin\dotnet-trace\Debug\net8.0\dotnet-trace.exe collect-linux
The collect-linux command is only supported on Linux.

dotnet-trace list-profiles

`list-profiles`

dotnet-trace profiles:
        dotnet-common                        - Lightweight .NET runtime diagnostics designed to stay low overhead.
                                               Includes:
                                                   GC
                                                   AssemblyLoader
                                                   Loader
                                                   JIT
                                                   Exceptions
                                                   Threading
                                                   JittedMethodILToNativeMap
                                                   Compilation
                                               Equivalent to --providers "Microsoft-Windows-DotNETRuntime:0x100003801D:4".
        dotnet-sampled-thread-time (collect) - Samples .NET thread stacks (~100 Hz) toestimate how much wall clock time code is using.
        gc-verbose                           - Tracks GC collections and samples object allocations.
        gc-collect                           - Tracks GC collections only at very low overhead.
        database                             - Captures ADO.NET and Entity Framework database commands
        cpu-sampling (collect-linux)         - Kernel CPU sampling events for measuring CPU usage.
        thread-time (collect-linux)          - Kernel thread context switch events for measuring CPU usage and wall clock time

[noahfalk]

This is looking pretty good. Aside from some small comments inline a few broader things:

1. Historically we relied on manual testing to keep these tools operating well but now that we have less time available from the testers I think we need to improve our automated testing. Partly this is to ensure we aren't inadvertently changing the original 'collect' verb and partly to ensure going forward the 'collect-linux' behavior doesn't regress either. I think the best way to do this would be:

• Open a new PR that we'll check in first containing some basic tests of the existing collect verb.
• Commit this PR 2nd and all the tests in the 1st PR should continue to pass. This ensures we didn't change anything unintended.

To do the testing we probably need to create some small interface shims. We already have an IConsole interface defined that could be moved to the shared Common folder. We could also create a small interface around the DiagnosticsClient.EventPipeCollect() API so that a test can return some dummy data in a stream instead. dotnet-counters has some example tests that show how we can run some code and then confirm the console output is what we expect. In this case I imagine we'd be running the Collect() function and giving some chosen input arguments.

2. I think there is a bit more adjustment to be done on some of the output text, but it should be fine to get this one in first, then tweak afterwards in some 3rd PR.

[mdh1418]

The failure is unrelated I think it's dotnet/runtime#120317 even after the refactor #5585

[noahfalk]

If testing caused multiple instances of CollectLinuxCommandHandler to run in parallel, would we want those instances to be sharing any of these static fields? I'm guessing we'd want all these fields to be instanced too.

[mdh1418]

Changed to non-static. From some learning, I'm seeing that parallelization occurs at the test class level, so it doesn't seem like the theory members would run in parallel.

[noahfalk]

Shouldn't there be some output about where the test output is being written to and telling the user to press Enter to stop the trace?

[mdh1418]

I added the output file to Collect-linux, thanks for the catch. As per the press enter/ctrl+c message, normally that is part of the status printing.

For collect-linux, I added the line rewriting/status printing to the callback, so to mock that for the tests, extended the RecordTraceInvoker test seam to invoke the callback once with the progress output type, causing the status+ Press Enter/Ctrl+C to emit once.