Description
The current test coverage configurations could be improved for more meaningful insights.
Integration tests, such as test_cli, often inflate coverage metrics by traversing code without thoroughly testing individual components. While integration tests are valuable, unit tests are essential for independently verifying the behaviour of individual code units. For example:, the utils.py has just one unit test!
To ensure that the test coverage is meaningful, the test-measurement should primarily be driven by unit tests, which allow precise control over test scenarios and edge cases. Furthermore, the test-files itself must be excluded from the coverage measurement.
I propose separating unit and integration tests, using markers and/or directories for clear differentiation. While unit tests should be the primary source for coverage metrics, integration tests can still provide supplementary insights into overall code traversal. This balanced approach ensures both high-quality tests and reliable coverage metrics.
I would like to take up this issue.
Ref: #76 #521
Would love to know your comments @vinayak-mehta @bosd