feat(polluter): add polluter bisect script #16235
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Sometimes tests from the
igniteui-angular
project fail when run as part of the full suite, even though they pass when executed in isolation.The reason is that some tests pollute the testing environment (e.g. by leaving behind global state or async handles), which causes later tests to behave incorrectly.
With a suite of ~200 test files and more than 5000 tests, it’s not feasible to manually track down the culprit file.
This PR introduces a polluter-bisect script that streamlines the process of identifying polluting tests by:
before
- only considers tests that run before the sentinel. (default)all
- considers all tests in the suite and runs the sentinel last.This makes it possible to isolate polluting test files much faster than running the entire suite repeatedly.
The script is not intended to run in CI; it’s a developer tool to aid in diagnosing flaky tests.
Usage
From the root of the repo, run:
The script will iteratively run subsets of tests until it identifies the polluting test file that causes the sentinel to fail.
Additional information (check all that apply):
Checklist:
feature/README.MD
updates for the feature docsREADME.MD
CHANGELOG.MD
updates for newly added functionalityng update
migrations for the breaking changes (migrations guidelines)