Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SDTEST-997] parallel tests gem support #299

Merged
merged 29 commits into from
Mar 24, 2025

Conversation

anmarchenko
Copy link
Member

@anmarchenko anmarchenko commented Mar 12, 2025

What does this PR do?
Adds proper support for the parallel_tests gem (when used with RSpec): now when running tests with parallel_rspec command datadog-ci will trace and report a single test session. Also, the worker processes will now share the remote configuration and other backend data between them, in order to save a lot of time and fire fewer requests to the Datadog's backend.

Motivation
When parallel_tests gem is used, we want to report a single test session to correctly calculate time and cost savings from Datadog Test Optimization product

Additional Notes
The most important changes are:

  • new parallel_tests subfolder in contrib: it instruments ParallelTests::CLI class to start test session and test module when CLI is invoked
  • new internal configuration test_visibility_drb_server_uri that is used to propagate DRb (distributed ruby) socket URI between worker processes via ENV variable
  • rspec instrumentation no longer stops test session and test module if it is running under parallel_tests
  • datadog-ci components (Remote, TestVisibility, TestManagement, TestOptimisation) now dump their global state to a file if started under parallel_tests. Worker processes then read this state from a file to avoid going to Datadog backend unnecessary. See utils/file_storage.rb file for the implementation of this storage interface. See utils/stateful.rb for a concern that adds ability to store internal state to any component.
  • introduced a new test suite tracing mode for test visibility: local_test_suites - when enabled, the test suites are traced in the local process (worker). This mode is enabled by default. Minitest instrumentation, on the other hand, always disables this mode because minitest split tests by individual test, not by test suite (test file). This is done for performance reasons because IPC increases the overhead of tracing.
  • test session context and test module context are now copied from remote process to each of the worker process. It includes span ID, name, test service and some tags that are shared between all span levels.

How to test the change?
Manual testing using anmarchenko/rubocop project, results:
image

image

Added unit tests to make sure that ParallelTests::CLI opens test_session and test_module

@anmarchenko anmarchenko changed the title Anmarchenko/parallel tests gem support [SDTEST-997] parallel tests gem support Mar 12, 2025
@anmarchenko anmarchenko force-pushed the anmarchenko/parallel_tests_gem_support branch from f28058e to 024fadd Compare March 18, 2025 14:39
@anmarchenko anmarchenko marked this pull request as ready for review March 18, 2025 15:01
@anmarchenko anmarchenko requested review from a team as code owners March 18, 2025 15:01
@anmarchenko anmarchenko requested a review from gnufede March 18, 2025 15:01
@anmarchenko anmarchenko merged commit ab9f163 into main Mar 24, 2025
196 checks passed
@anmarchenko anmarchenko deleted the anmarchenko/parallel_tests_gem_support branch March 24, 2025 14:19
@github-actions github-actions bot added this to the 1.15.0 milestone Mar 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants