Skip to content

Changes in the FV3 dycore to support 3DTKE EDMF PBL physic scheme #382

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
laurenchilutti merged 5 commits into from
Jun 30, 2025
Merged

Changes in the FV3 dycore to support 3DTKE EDMF PBL physic scheme #382

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
7bdc2c6
3D-TKE EMDF GFS PBL scheme related changes from FIU (Ping Zhu, Ping.Z…
BinLiu-NOAA May 9, 2025
c50e339
Update submodules fv3/atmos_cubed_sphere for 3dtke data structure
May 23, 2025
bd0e414
Update based on stylistic comments and mpi use in dyn_core
Jun 3, 2025
219f701
Merge remote-tracking branch 'remotes/upstream/dev/emc' into feature/…
BinLiu-NOAA Jun 13, 2025
ec21a2a
Change dku3d_h to be an optional input variable for d_sw subroutine. …
BinLiu-NOAA Jun 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 58 additions & 2 deletions driver/UFS/atmosphere.F90
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,8 @@ module atmosphere_mod
#ifdef GFS_TYPES
use GFS_typedefs, only: IPD_control_type => GFS_control_type, kind_phys
use GFS_typedefs, only: GFS_statein_type, GFS_stateout_type, GFS_sfcprop_type
! SA-3D-TKE (kyf)
use GFS_typedefs, only: GFS_tbd_type
#else
use IPD_typedefs, only: IPD_data_type, IPD_control_type, kind_phys => IPD_kind_phys
#endif
Expand Down Expand Up @@ -693,11 +695,16 @@ subroutine atmosphere_dynamics ( Time )
! Atm(n)%flagstruct%n_split, Atm(n)%flagstruct%q_split, &
Atm(n)%u, Atm(n)%v, Atm(n)%w, Atm(n)%delz, &
Atm(n)%flagstruct%hydrostatic, &
!The following variable is used for SA-3D-TKE
Atm(n)%flagstruct%sa3dtke_dyco, &
Atm(n)%pt , Atm(n)%delp, Atm(n)%q, Atm(n)%ps, &
Atm(n)%pe, Atm(n)%pk, Atm(n)%peln, &
Atm(n)%pkz, Atm(n)%phis, Atm(n)%q_con, &
Atm(n)%omga, Atm(n)%ua, Atm(n)%va, Atm(n)%uc, &
Atm(n)%vc, Atm(n)%ak, Atm(n)%bk, Atm(n)%mfx, &
Atm(n)%vc, &
!The following variable is used for SA-3D-TKE (kyf) (modify for data structure)
Atm(n)%sa3dtke_var, &
Atm(n)%ak, Atm(n)%bk, Atm(n)%mfx, &
Atm(n)%mfy , Atm(n)%cx, Atm(n)%cy, Atm(n)%ze0, &
Atm(n)%flagstruct%hybrid_z, &
Atm(n)%gridstruct, Atm(n)%flagstruct, &
Expand Down Expand Up @@ -1894,9 +1901,13 @@ subroutine adiabatic_init(zvir,nudge_dz,time)
Atm(mygrid)%ptop, Atm(mygrid)%ks, nq, n_split_loc, &
Atm(mygrid)%flagstruct%q_split, Atm(mygrid)%u, Atm(mygrid)%v, Atm(mygrid)%w, &
Atm(mygrid)%delz, Atm(mygrid)%flagstruct%hydrostatic, &
!The following variable is used for SA-3D-TKE
Atm(mygrid)%flagstruct%sa3dtke_dyco, &
Atm(mygrid)%pt, Atm(mygrid)%delp, Atm(mygrid)%q, Atm(mygrid)%ps, &
Atm(mygrid)%pe, Atm(mygrid)%pk, Atm(mygrid)%peln, Atm(mygrid)%pkz, Atm(mygrid)%phis, &
Atm(mygrid)%q_con, Atm(mygrid)%omga, Atm(mygrid)%ua, Atm(mygrid)%va, Atm(mygrid)%uc, Atm(mygrid)%vc, &
!The following variable is used for SA-3D-TKE (kyf) (modify for data structure)
Atm(mygrid)%sa3dtke_var, &
Atm(mygrid)%ak, Atm(mygrid)%bk, Atm(mygrid)%mfx, Atm(mygrid)%mfy, &
Atm(mygrid)%cx, Atm(mygrid)%cy, Atm(mygrid)%ze0, Atm(mygrid)%flagstruct%hybrid_z, &
Atm(mygrid)%gridstruct, Atm(mygrid)%flagstruct, &
Expand All @@ -1909,9 +1920,13 @@ subroutine adiabatic_init(zvir,nudge_dz,time)
Atm(mygrid)%ptop, Atm(mygrid)%ks, nq, n_split_loc, &
Atm(mygrid)%flagstruct%q_split, Atm(mygrid)%u, Atm(mygrid)%v, Atm(mygrid)%w, &
Atm(mygrid)%delz, Atm(mygrid)%flagstruct%hydrostatic, &
!The following variable is used for SA-3D-TKE
Atm(mygrid)%flagstruct%sa3dtke_dyco, &
Atm(mygrid)%pt, Atm(mygrid)%delp, Atm(mygrid)%q, Atm(mygrid)%ps, &
Atm(mygrid)%pe, Atm(mygrid)%pk, Atm(mygrid)%peln, Atm(mygrid)%pkz, Atm(mygrid)%phis, &
Atm(mygrid)%q_con, Atm(mygrid)%omga, Atm(mygrid)%ua, Atm(mygrid)%va, Atm(mygrid)%uc, Atm(mygrid)%vc, &
!The following three variables are used for SA-3D-TKE (kyf) (modify for data structure)
Atm(mygrid)%sa3dtke_var, &
Atm(mygrid)%ak, Atm(mygrid)%bk, Atm(mygrid)%mfx, Atm(mygrid)%mfy, &
Atm(mygrid)%cx, Atm(mygrid)%cy, Atm(mygrid)%ze0, Atm(mygrid)%flagstruct%hybrid_z, &
Atm(mygrid)%gridstruct, Atm(mygrid)%flagstruct, &
Expand Down Expand Up @@ -1985,9 +2000,13 @@ subroutine adiabatic_init(zvir,nudge_dz,time)
Atm(mygrid)%ptop, Atm(mygrid)%ks, nq, Atm(mygrid)%flagstruct%n_split, &
Atm(mygrid)%flagstruct%q_split, Atm(mygrid)%u, Atm(mygrid)%v, Atm(mygrid)%w, &
Atm(mygrid)%delz, Atm(mygrid)%flagstruct%hydrostatic, &
!The following variable is used for SA-3D-TKE
Atm(mygrid)%flagstruct%sa3dtke_dyco, &
Atm(mygrid)%pt, Atm(mygrid)%delp, Atm(mygrid)%q, Atm(mygrid)%ps, &
Atm(mygrid)%pe, Atm(mygrid)%pk, Atm(mygrid)%peln, Atm(mygrid)%pkz, Atm(mygrid)%phis, &
Atm(mygrid)%q_con, Atm(mygrid)%omga, Atm(mygrid)%ua, Atm(mygrid)%va, Atm(mygrid)%uc, Atm(mygrid)%vc, &
!The following three variables are used for SA-3D-TKE (kyf) (modify for data structure)
Atm(mygrid)%sa3dtke_var, &
Atm(mygrid)%ak, Atm(mygrid)%bk, Atm(mygrid)%mfx, Atm(mygrid)%mfy, &
Atm(mygrid)%cx, Atm(mygrid)%cy, Atm(mygrid)%ze0, Atm(mygrid)%flagstruct%hybrid_z, &
Atm(mygrid)%gridstruct, Atm(mygrid)%flagstruct, &
Expand All @@ -1999,9 +2018,13 @@ subroutine adiabatic_init(zvir,nudge_dz,time)
Atm(mygrid)%ptop, Atm(mygrid)%ks, nq, Atm(mygrid)%flagstruct%n_split, &
Atm(mygrid)%flagstruct%q_split, Atm(mygrid)%u, Atm(mygrid)%v, Atm(mygrid)%w, &
Atm(mygrid)%delz, Atm(mygrid)%flagstruct%hydrostatic, &
!The following variable is used for SA-3D-TKE
Atm(mygrid)%flagstruct%sa3dtke_dyco, &
Atm(mygrid)%pt, Atm(mygrid)%delp, Atm(mygrid)%q, Atm(mygrid)%ps, &
Atm(mygrid)%pe, Atm(mygrid)%pk, Atm(mygrid)%peln, Atm(mygrid)%pkz, Atm(mygrid)%phis, &
Atm(mygrid)%q_con, Atm(mygrid)%omga, Atm(mygrid)%ua, Atm(mygrid)%va, Atm(mygrid)%uc, Atm(mygrid)%vc, &
!The following three variables are used for SA-3D-TKE (kyf) (modify for data structure)
Atm(mygrid)%sa3dtke_var, &
Atm(mygrid)%ak, Atm(mygrid)%bk, Atm(mygrid)%mfx, Atm(mygrid)%mfy, &
Atm(mygrid)%cx, Atm(mygrid)%cy, Atm(mygrid)%ze0, Atm(mygrid)%flagstruct%hybrid_z, &
Atm(mygrid)%gridstruct, Atm(mygrid)%flagstruct, &
Expand Down Expand Up @@ -2064,16 +2087,19 @@ end subroutine adiabatic_init
!>@detail Performs a mass adjustment to be consistent with the
!! GFS physics and if necessary, converts quantities to hydrostatic
!! representation.
!! SA-3D-TKE (added IPD_Tbd in the input) (kyf)
#if defined(OVERLOAD_R4)
#define _DBL_(X) DBLE(X)
#define _RL_(X) REAL(X,KIND=4)
#else
#define _DBL_(X) X
#define _RL_(X) X
#endif
subroutine atmos_phys_driver_statein (IPD_Control, IPD_Statein, Atm_block,flip_vc)
subroutine atmos_phys_driver_statein (IPD_Control, IPD_Statein, IPD_Tbd, Atm_block,flip_vc)
type (IPD_control_type), intent(in) :: IPD_Control
type (GFS_statein_type), intent(inout) :: IPD_Statein
! SA-3D-TKE (added IPD_Tbd) (kyf)
type (GFS_tbd_type), intent(inout) :: IPD_Tbd
type (block_control_type), intent(in) :: Atm_block
logical, intent(in) :: flip_vc
!--------------------------------------
Expand Down Expand Up @@ -2113,8 +2139,10 @@ subroutine atmos_phys_driver_statein (IPD_Control, IPD_Statein, Atm_block,flip_v
!---------------------------------------------------------------------
! use most up to date atmospheric properties when running serially
!---------------------------------------------------------------------
! SA-3D-TKE added IPD_Tbd (kyf)
!$OMP parallel do default (none) &
!$OMP shared (Atm_block, Atm, IPD_Control, IPD_Statein, npz, nq, ncnst, sphum, liq_wat, &
!$OMP IPD_Tbd, &
!$OMP ice_wat, rainwat, snowwat, graupel, pk0inv, ptop, &
!$OMP pktop, zvir, mygrid, dnats, nq_adv, flip_vc) &
#ifdef MULTI_GASES
Expand All @@ -2130,6 +2158,27 @@ subroutine atmos_phys_driver_statein (IPD_Control, IPD_Statein, Atm_block,flip_v
! log(pe) <-- prsik

blen = Atm_block%blksz(nb)
!The following is for SA-3D-TKE
if(Atm(mygrid)%flagstruct%sa3dtke_dyco) then
!The following is for SA-3D-TKE (pass dku to dyn_core)
do k = 1, npz
kz = npz+1-k
if(flip_vc) then
k1 = kz ! flipping the index
else
k1 = k
endif
do ix = 1, blen
i = Atm_block%index(nb)%ii(ix)
j = Atm_block%index(nb)%jj(ix)
! SA-3D-TKE (added im) (kyf)
im = IPD_control%chunk_begin(nb)+ix-1
! SA-3D-TKE (modified IPD_Tbd and ix into im) (kyf) (modify for data structure)
Atm(mygrid)%sa3dtke_var%dku3d_h(i,j,k) = IPD_Tbd%dku3d_h(im,k1)
Atm(mygrid)%sa3dtke_var%dku3d_e(i,j,k) = IPD_Tbd%dku3d_e(im,k1)
Comment on lines +2167 to +2168
Copy link
Contributor

@bensonr bensonr May 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logic is copying the dku3d_h and dku3d_e from the physics state (IPD_tbd) into the dynamics memory space (Atm) after the dycore has run, but before the atmospheric physics are computed. Is this seeming lagging of data for dku3d*_ the intended behavior? If so, please add some comments explaining why this logic is the way it is here.

Copy link

@zhup01 zhup01 May 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the comments! This is a great question. dku3d_h and dku3d_e are the horizontal eddy diffusivities for momentum and scalars, which are used for calculating damping coefficient and horizontal TKE transport in the dycore. dku3d_h and dku3d_e are the functions of TKE that this determined by the TKE budget equation in the atmospheric physics. So, here is the thing. If they are put before the atmospheric physics, it means that dku3d_h and dku3d_e determined by the previous timestep TKE is used for calculating TKE transport that feeds into the atmospheric physics for updating the next timestep TKE. The updated TKE is then used for updating dku3d_h and dku3d_e for the next timestep. If they are put after the dku3d_h and dku3d_e, the TKE is updated but dku3d_h and dku3d_e are still the ones determined by the last timestep TKE since they are caclulated before TKE updating. So, the difference would be negligible. Of course, we may transfer dku3d_h and dku3d_e after the atmospheric phyics. But we don't know how to do this in the dycore.

enddo
enddo
endif

do k = 1, npz
!Indices for FV's vertical coordinate, for which 1 = top
Expand All @@ -2154,6 +2203,13 @@ subroutine atmos_phys_driver_statein (IPD_Control, IPD_Statein, Atm_block,flip_v
endif
IPD_Statein%vvl(im,k) = _DBL_(_RL_(Atm(mygrid)%omga(i,j,k1)))
IPD_Statein%prsl(im,k) = _DBL_(_RL_(Atm(mygrid)%delp(i,j,k1))) ! Total mass
!The following is for SA-3D-TKE (kyf) (modify for data structure)
if(Atm(mygrid)%flagstruct%sa3dtke_dyco) then
IPD_Statein%def_1(ix,k) = _DBL_(_RL_(Atm(mygrid)%sa3dtke_var%deform_1(i,j,k1)))
IPD_Statein%def_2(ix,k) = _DBL_(_RL_(Atm(mygrid)%sa3dtke_var%deform_2(i,j,k1)))
IPD_Statein%def_3(ix,k) = _DBL_(_RL_(Atm(mygrid)%sa3dtke_var%deform_3(i,j,k1)))
endif

if (Atm(mygrid)%flagstruct%do_skeb) IPD_Statein%diss_est(im,k) = _DBL_(_RL_(Atm(mygrid)%diss_est(i,j,k1)))

if(flip_vc) then
Expand Down
Loading