Currently, I locally use Pixi which is a modern project-oriented Conda environment manager, which I would like to contribute here.

It can easily manage multiple environment and tasks (along with task dependencies).
I have it setup so that running pixi run -e clang-17 test debug-avx will run an avx build with clang-17 compiler from scratch.
That is, this command is all it takes to get stated, from there dependencies download, env creation, building are all done automatically (with caching).

It's extremely convenient to quickly test any configuration, and very friendly to beginners.
Of course, having such manifest file does not mean all developers have to use it, the project keep working without.

As discussed with @JohanMabille