Assess how to reduce testing computational costs
Right now, CI testing takes >30 mins, maybe more for Windows machiens.
It is fantastic that we have such exhaustive tests, but there's a chance we have some redundant or unnecessary tests that lead to unnecessarily inflated CI runtimes.
We should:
- assess if the existing tests can be intelligently combined to reduce computational burden
- potentially look at breaking up unit, regression, and integration tests into different CI pipelines so they run in parallel
- reduce size and scope of examples we test
- reduce model fidelity where reasonable
Assess how to reduce testing computational costs
Right now, CI testing takes >30 mins, maybe more for Windows machiens.
It is fantastic that we have such exhaustive tests, but there's a chance we have some redundant or unnecessary tests that lead to unnecessarily inflated CI runtimes.
We should: