Support pfunit testing with Fab

[hiker]

Version

main

Are there any linked Issues or Pull Requests?

#279

What happened?

Add support for compiling unit tests to the Fab scripts. That's one part of #279 .

Relevant log output

[hiker]

During the review of #246 support for building pfunit with Fab was discussed. Copying the relevant discussion here for a follow-up PR:

@MatthewHambley wrote:

The concept of "debug" and "production" makes no sense for the tests themselves. I can see an argument for trying different compiler profiles with the units under test, but that is not how things work at the moment.

For the time being I think we should replicate the current approach, which is fixed compiler profile for testing. As long as we don't close off the possibility of different compiler profiles for tests and test articles in the future, we can consider that later.

[hiker]

As a quick update; I have started to introduce an additional base class, which adds support for unit testing (but which can also be used in case that an app has no unit tests). The advantage of this approach is that all pfUnit/testing related changes are contained in one file (instead of making the existing base class, which is already over 430 lines, larger). This new class has a bit under 200 lines.

Testing, and verification that it integrates with lfric_apps is still outstanding.

One current drawback: till we have a new Fab release which supports path-specific flags better, even the normal application will get linked with pfunit (and pre-processed with TEST_SUITES being defined). That doesn't really matter, and I don't want to waste time using the 'old' way (feeding two different kind of flags into the steps), when there is a new one that combines path-independent and path-specific flags into a single argument (and e.g. the current link_exe step does not even support path-specific flags at all)

We need to discuss the usage of compilation modes as indicated above. I haven't added separate unit-test flags (which is what the current build system uses), since I think it's much more useful to run tests with compilation settings that are actually used (plus Fab does not easily allow a file being compiled with two different set of flags, meaning we would need to setup a complete separate workspace for unit-testing only, which might also mean that many files need to be compiled twice).

It is rather trivial to add unit-test as a separate compilation profile (and then disabling the creation of the application binary, and instead only creating the testing binary), if this should really be the long term vision here.