Skip to content

Conversation

IMinchev64
Copy link
Contributor

@IMinchev64 IMinchev64 commented Sep 23, 2025

Description

Sometimes tests from the igniteui-angular project fail when run as part of the full suite, even though they pass when executed in isolation.
The reason is that some tests pollute the testing environment (e.g. by leaving behind global state or async handles), which causes later tests to behave incorrectly.

With a suite of ~200 test files and more than 5000 tests, it’s not feasible to manually track down the culprit file.

This PR introduces a polluter-bisect script that streamlines the process of identifying polluting tests by:

  • Allowing you to specify a sentinel test file (the test known to fail in polluted environments).
  • Running a binary search over the rest of the suite to narrow down the minimal set of files that trigger the sentinel failure.
  • Supporting two modes:
    • before - only considers tests that run before the sentinel. (default)
    • all - considers all tests in the suite and runs the sentinel last.
  • Providing a flag to optionally skip the initial full-set scan if you already know the sentinel fails in the suite.
  • Generating a temporary polluter-runner spec file to enforce deterministic execution order (bypassing Karma’s automatic sorting).

This makes it possible to isolate polluting test files much faster than running the entire suite repeatedly.
The script is not intended to run in CI; it’s a developer tool to aid in diagnosing flaky tests.

Usage

From the root of the repo, run:

# Default: search only in files before the sentinel
npm run polluter:bisect -- sentinel-file.spec.ts before

# Search across all test files, with sentinel always last
npm run polluter:bisect -- sentinel-file.spec.ts all

# Skip the initial full-set scan (faster if you already know the sentinel fails in the suite)
npm run polluter:bisect -- sentinel-file.spec.ts before --skip-initial

The script will iteratively run subsets of tests until it identifies the polluting test file that causes the sentinel to fail.

NOTE:
In order for the script to work correctly you should set only a single test executor in projects/igniteui-angular/karma.conf.js under parallelOptions.

Additional information (check all that apply):

  • Bug fix
  • New functionality
  • Documentation
  • Demos
  • CI/CD

Checklist:

  • All relevant tags have been applied to this PR
  • This PR includes unit tests covering all the new code (test guidelines)
  • This PR includes API docs for newly added methods/properties (api docs guidelines)
  • This PR includes feature/README.MD updates for the feature docs
  • This PR includes general feature table updates in the root README.MD
  • This PR includes CHANGELOG.MD updates for newly added functionality
  • This PR contains breaking changes
  • This PR includes ng update migrations for the breaking changes (migrations guidelines)
  • This PR includes behavioral changes and the feature specification has been updated with them

@IMinchev64 IMinchev64 marked this pull request as ready for review September 24, 2025 07:31
@IMinchev64 IMinchev64 added the ❌ status: awaiting-test PRs awaiting manual verification label Sep 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
❌ status: awaiting-test PRs awaiting manual verification
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants