!perf: introduce performance analysis subsystem

[adamgreloch]

JIRA: RTOS-1057

Introduces a subsystem intended for various performance analysis
mechanisms: tracing, sample-based methods, PMU utilization, etc.

Description

Breaking changes:

• This commit generalizes the perf_* syscalls to support various perf
  modes
• Previous perf semantics are removed
• The return value of perf_start on success changes from 0 to a
  non-negative number denoting count of channels that shouold be read by
  perf_read.
• perf_read accepts additional chan argument denoting an ID of a
  channel to be read (a mode could serve multiple read-only channels)

The subsystem currently implements perf_mode_trace mode - a mechanism
for tracing syscalls, interrupts, locks and scheduler events. The events
are outputed in CTF format to a set of cbuffers that can be retrieved
from userspace via perf_read(perf_mode_trace, ..., chan) and later
processed by 3rd party libraries like babeltrace2.

This commit also adds a perf_stop syscall fo stopping the perf action
without freeing the internal buffers, to allow reading them post-factum
and not disturb the trace with perf_read calls while it is being
recorded.

The trace can be gathered in start-stop fashion or in a rolling-window
fashion, if perf_start is supplied with PERF_TRACE_FLAG_ROLLING flag.
In the rolling-window fashion, the trace buffer write ops will discard
the oldest events if the buffers are full.

Motivation and Context

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Chore (refactoring, style fixes, git/CI config, submodule management, no code logic changes)

How Has This Been Tested?

• Already covered by automatic testing.
• New test added: (add PR link here).
• Tested by hand on: ia32-generic-qemu

Checklist:

• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have added tests to cover my changes.
• All new and existing linter checks and tests passed.
• My changes generate no new compilation warnings for any of the targets.

Special treatment

• This PR needs additional PRs to work (list the PRs, preferably in merge-order).
  !perf: generalize perf syscalls libphoenix#415
• I will merge this PR by myself when appropriate.

[gemini-code-assist]

Hello @adamgreloch, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

Summary of Changes

Hello team, gemini-code-assist here to provide a summary of this pull request. This PR introduces a new performance analysis subsystem to the kernel. The primary goal is to provide a framework for various performance monitoring mechanisms, starting with detailed kernel event tracing. A significant change is the generalization of the existing perf_* syscalls to support different modes of operation, including the previously existing thread-based performance monitoring and the new tracing mode. The tracing mode captures events like syscall entries/exits, interrupt handling, lock operations, and scheduler activity, writing them in CTF format to an in-memory circular buffer that can be read from userspace.

Highlights

• New Performance Subsystem: Introduces a dedicated subsystem (perf/) for performance analysis tools.
• Generalized perf_* Syscalls: The existing perf_start, perf_read, and perf_finish syscalls are modified to accept a perf_mode_t argument, allowing them to control different performance monitoring modes. The previous functionality is preserved under perf_mode_threads.
• Kernel Event Tracing (perf_mode_trace): Adds a new tracing mode that captures various kernel events (syscalls, interrupts, locks, scheduler) and outputs them in Common Trace Format (CTF).
• CTF Output and In-Memory Buffer: Trace events are written to a 4MB in-memory circular buffer using a new buffer implementation (perf/buffer-mem.c). The data format conforms to the CTF metadata defined in perf/tsdl/metadata.
• Instrumentation: Instrumentation points are added in key kernel areas (interrupt dispatcher, scheduler, thread creation/termination, lock operations, syscall dispatcher) to record tracing events.

Changelog

Click here to see the changelog

• Makefile
  • Includes the new perf/Makefile to integrate the performance subsystem into the build.
• hal/ia32/interrupts.c
  • Includes perf/events.h.
  • Adds trace_irqs flag to control interrupt tracing.
  • Adds calls to perf_traceEventsInterruptEnter (line 201) and perf_traceEventsInterruptExit (line 217) within interrupts_dispatchIRQ when tracing is enabled.
  • Adds _hal_interruptsTrace function (line 456) to enable/disable interrupt tracing.
  • Initializes trace_irqs to 0 in _hal_interruptsInit (line 467).
• hal/interrupts.h
  • Declares the new _hal_interruptsTrace function (line 35).
• include/perf.h
  • New file defining the perf_mode_t enum (line 20) for different performance modes.
• main.c
  • Includes perf/perf.h.
  • Calls _perf_init (line 112) during kernel initialization.
• perf/Makefile
  • New file defining the object files for the performance subsystem (perf.o, trace.o, buffer-mem.o).
• perf/buffer-mem.c
  • New file implementing an in-memory circular buffer for trace data.
  • Includes functions for buffer initialization (trace_bufferInit, line 131), starting (_trace_bufferStart, line 81), finishing (_trace_bufferFinish, line 109), reading (_trace_bufferRead, line 97), and writing (_trace_bufferWrite, line 103).
  • Uses kernel VM functions (vm_mapFind, vm_pageAlloc, page_map, vm_munmap, vm_pageFree) to manage the buffer memory.
• perf/buffer.h
  • New file declaring the interface for the trace event buffer.
• perf/events.h
  • New file defining CTF event IDs (line 23) for various kernel events.
  • Provides inline helper functions (e.g., perf_traceEventsLockSet, perf_traceEventsInterruptEnter, perf_traceEventsThreadCreate, perf_traceEventsSyscallEnter, perf_traceEventsSchedEnter) to write specific event data to the trace buffer using perf_traceEventsWrite.
• perf/perf.c
  • New file implementing the main performance subsystem interface.
  • Includes trace.h and perf.h.
  • Implements _perf_init (line 21), perf_start (line 27), perf_read (line 40), and perf_finish (line 53), dispatching calls based on the perf_mode_t argument.
• perf/perf.h
  • New file declaring the main performance subsystem interface functions.
• perf/trace.c
  • New file implementing the CTF tracing backend.
  • Includes buffer.h, events.h, and trace.h.
  • Implements _perf_traceInit (line 152), perf_traceStart (line 82), perf_traceRead (line 108), perf_traceFinish (line 125), and perf_traceIsRunning (line 76).
  • Handles writing events with timestamps (perf_traceEventsWrite, line 53) using the trace buffer.
  • Includes logic for detecting timer non-monotonicity and buffer overflows.
• perf/trace.h
  • New file declaring the interface for the trace backend.
• perf/tsdl/metadata
  • New file defining the CTF metadata for the kernel trace stream.
  • Defines data types, clock information, stream structure, and the fields for each traced event type (lock, interrupt, thread, syscall, scheduler).
• proc/process.c
  • Includes perf/events.h.
  • Adds process_getName function (line 1730) to retrieve a process's name from its path or arguments.
• proc/process.h
  • Declares the new process_getName function (line 82).
• proc/threads.c
  • Includes perf/events.h.
  • Adds calls to thread-related trace event functions (perf_traceEventsThreadScheduling, etc.) in the _perf_event helpers (lines 148, 155, 162, 169).
  • Renames the old perf_start, perf_read, perf_finish functions to perf_threadsStart (line 339), perf_threadsRead (line 368), and perf_threadsFinish (line 380) respectively.
  • Adds calls to perf_traceEventsSchedEnter (line 680) and perf_traceEventsSchedExit (line 686) in threads_schedule.
  • Adds calls to perf_traceEventsThreadCreate (line 839) in proc_threadCreate and perf_traceEventsThreadEnd (line 981) in proc_threadEnd.
  • Adds calls to perf_traceEventsLockSet (line 1584) in _proc_lockSet and perf_traceEventsLockClear (line 1691) in _proc_lockUnlock.
  • Updates proc_threadsList (line 1937) to use the new process_getName function (line 1982).
• proc/threads.h
  • Renames the declarations for the old perf_* functions to perf_threadsStart (line 97), perf_threadsRead (line 100), and perf_threadsFinish (line 103).
• syscalls.c
  • Includes new performance-related headers (include/perf.h, perf/perf.h, perf/trace.h, perf/events.h).
  • Modifies syscalls_perf_start (line 489), syscalls_perf_read (line 503), and syscalls_perf_finish (line 520) to accept the perf_mode_t argument.
  • Adds calls to perf_traceEventsSyscallEnter (line 1915) and perf_traceEventsSyscallExit (line 1919) in syscalls_dispatch.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

---

A kernel's heart, a hidden beat,
Now traced in time, a data treat.
From syscall's call to IRQ's flight,
Performance shines in new-found light.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a comprehensive performance analysis subsystem, which is a significant and valuable addition. The generalization of existing perf_* syscalls and the implementation of CTF-based event tracing are well-executed. The code is generally clear, well-structured, and adheres to common C kernel development practices.

I've noted a couple of areas for improvement below, primarily concerning code maintainability and a performance consideration that aligns with an existing TODO. Given the scope and impact of these changes, especially the breaking change to syscalls, thorough testing is crucial. While manual testing is noted, considering automated tests for this new subsystem would be beneficial for long-term stability and maintenance.

Summary of Findings

• Code Duplication: The _trace_bufferFinish function in perf/buffer-mem.c duplicates logic from trace_bufferFree for deallocating buffer memory. This could be refactored to call trace_bufferFree.
• Performance of Lock Event Tracing: Copying lock names on every lock/unlock event in perf/events.h can be inefficient. An existing TODO comment rightly points this out, suggesting a more optimized approach.
• Unused struct members (Low Severity - Not directly commented): The buffer_common struct in perf/buffer-mem.c contains lastTimestamp and prev fields (lines 27, 31) that are not used within that file. Similarly, trace_common.lastTimestamp in perf/trace.c (line 21) appears unused. These might be leftovers and could be removed if truly unnecessary to avoid confusion.

Merge Readiness

This pull request introduces a significant and valuable performance analysis subsystem. The changes are generally well-implemented. However, there are a couple of medium-severity issues related to code duplication and a performance consideration (already noted by a TODO) that should ideally be addressed before merging to improve maintainability and efficiency.

Given the breaking nature of the syscall changes and the introduction of a new subsystem, ensuring comprehensive testing (including consideration for automated tests) is highly recommended.

I am unable to approve pull requests directly. Please ensure these points are considered and further review is conducted if necessary before merging.

[etiaro]

A few minor suggestions, apart from that looks good.