Some more MPAS functionality

[dustinswales]

Description

Contains updates to the UFS interface to the MPAS dynamical core.

New advancements include:

• Constituent handling and scalar advection. New module, ufs_mpas_constituents, which contains interface code between MPAS constituents and the UFS.
• MPAS native output stream. Contained within the ufs_mpas_io module. This module also includes procedures for reading in the various MPAS input streams (e.g., invariant, input, sfc_input)
• Support for lateral boundary condition (LBC) stream. New module with code related to regional MPAS configurations, ufs_mpas_boundaries. The LBCs are updated during forecast integration, _run phase in ufs_mpas_subdriver.F90.
• Progress on CCPP physics coupling. New MPAS input stream (sfc_input) for gridded fields needed by the physics. Additional P2D coupling procedures to handle the state evolution between physics group and dycore calls. Added diagnostics to output stream.

Issue(s) addressed

• UWM #2782
• UWM #2879

Testing

On Ursa and Derecho with GNU.
Expanded the existing (barebones) MPAS UWM RTs to a) run 12-hour forecast w/ 3-hour output in RELEASE mode and b) added a new DEBUG test that runs a 3-hour forecast with hourly output.

Dependencies

ufs-community/ccpp-physics#365

[NickSzapiro-NOAA]

Is there anything else I need to address?

I haven't looked at the new baselines yet but wonder if it's worth comparing real or idealized test cases to MPAS-A standalone (if that's the reference implementation), thanks to the added transport and output.

[dustinswales]

Is there anything else I need to address?

I haven't looked at the new baselines yet but wonder if it's worth comparing real or idealized test cases to MPAS-A standalone (if that's the reference implementation), thanks to the added transport and output.

@NickSzapiro-NOAA I've looked at the baselines (e.g., T, u, v, q, tracers) and they seem realistic. For the next PR, when the physics is included and we have more diagnostics, I will include some figures from offline and inline MPAS.

[dustinswales]

@NickSzapiro-NOAA Thanks for the review. I addressed all of your comments.

[dustinswales]

@NickSzapiro-NOAA @DusanJovic-NOAA Here are some simple difference plots (Standalone MPAS - Inline (UFS) MPAS) of the state variables at the lowest model level.

[DeniseWorthen]

@dustinswales These differences must be at least at the round-off level, surely?

[dustinswales]

@dustinswales These differences must be at least at the round-off level, surely?

@DeniseWorthen I'm sure there is, but I was shocked that they looked this close just eye-balling it.
I'm going to dive a bit deeper and create better figures. Stay tuned...

[NickSzapiro-NOAA]

If it's just advection and output, shouldn't MPAS-A standalone match MPAS-A dycore in UFS exactly?

[dustinswales]

If it's just advection and output, shouldn't MPAS-A standalone match MPAS-A dycore in UFS exactly?

@NickSzapiro-NOAA My expectation is for initial agreement that diverges with forecast time. There are many differences between the modeling systems, so there's that piece of it. But I will also admit that I'm also not sure how a simulation w/o physics should evolve?

Here's a joint PDFs of global mean differences as a function of forecast time and model layer. For all fields there is initial agreement with subsequent drifting apart of the forecasts, especially aloft.

Theta:

qv:

uwind:

vwind:

[NickSzapiro-NOAA]

Maybe I'm wrong to assume that the dynamics should match but the physics will differ ... but it looks like differences start near the model top

Is it possible for you to share (here or separately) what you're running for this comparison @dustinswales ?
Is MPAS standalone using the regression test inputs too?

[dustinswales]

@NickSzapiro-NOAA @DusanJovic-NOAA I think I've gotten to the bottom of it (Some default nml settings were not correct).

Here are the jointPDFs after correcting the configuration:
inlineMPAS - standaloneMPAS (theta)

inlineMPAS - standaloneMPAS (v_wind)

inlineMPAS - standaloneMPAS (u_wind)

For comparison, here are the joint PDFs of standaloneMPAS(gnu) - standaloneMPAS(ifx):
(theta)

(v_wind)

(u_wind)

So inlineMPAS introduces less differences wrt to standaloneMPAS than does changing the compiler!

[gspetro-NOAA]

@dustinswales Could you revert .gitmodules and update the CCPP hash to 6904ad4?

[gspetro-NOAA]

This PR can be merged :)