Skip to content

Enable tropcy_qc jobs in realtime and fix bug in jjob_header.sh#4532

Draft
DavidHuber-NOAA wants to merge 8 commits intoNOAA-EMC:developfrom
DavidHuber-NOAA:feature/realtime_tropcy
Draft

Enable tropcy_qc jobs in realtime and fix bug in jjob_header.sh#4532
DavidHuber-NOAA wants to merge 8 commits intoNOAA-EMC:developfrom
DavidHuber-NOAA:feature/realtime_tropcy

Conversation

@DavidHuber-NOAA
Copy link
Contributor

@DavidHuber-NOAA DavidHuber-NOAA commented Feb 9, 2026

Description

This adds a Jinja-configurable variable to enable the tropcy_qc portion of the prep job in realtime cases. This also fixes a bug in jjob_header.sh that would cause the external setpdy.sh script to fail due to unset variable names.

Resolves #4531

Type of change

  • Bug fix (fixes something broken)
  • New feature (adds functionality)
  • Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

  • Is this change expected to change outputs YES
    • GFS
  • Is this a breaking change (a change in existing functionality)? NO
  • Does this change require a documentation update? NO
  • Does this change require an update to any of the following submodules? NO

How has this been tested?

Will test in a real-time parallel

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • (This cannot be tested in CI) This change is covered by an existing CI test or a new one has been added

@DavidHuber-NOAA
Copy link
Contributor Author

This is currently being tested in /lfs/h2/emc/ptmp/david.huber/rt_tropcy/EXP/tropcy_reloc for cycles 2026020900-2026020906.

@DavidHuber-NOAA DavidHuber-NOAA changed the title Enable tropcy_qc jobs in realtime Enable tropcy_qc jobs in realtime and fix bug in jjob_header.sh Feb 9, 2026
@DavidHuber-NOAA
Copy link
Contributor Author

If merged, this PR will enable the PROCESS_TROPCY portion of the prep job in real-time cases.

@DavidHuber-NOAA
Copy link
Contributor Author

Marking this PR as ready for review. The real-time test was able to run through the prep jobs. I will continue running that test through 2 cycles and post results when complete.

@DavidHuber-NOAA
Copy link
Contributor Author

Two full real-time cycles completed yesterday and both successfully executed the PROCESS_TROPCY portion of the prep job. Only the gfs cycle had tropical cyclone data to process and neither cycle had data to ingest into the analysis. Here are the lines specific to the gfs_anal job when reading the tcvitals file:

 READ_TCPS:  IANLDATE =   2026020912
 READ_TCPS:  IGNORE TC_VITALS ENTRY #            1    
 READ_TCPS:  MISMATCHED FROM ANALYSIS TIME, OBS / ANL DATES =   2026013018
  2026020912
 READ_TCPS:  IGNORE TC_VITALS ENTRY #            2    
 READ_TCPS:  MISMATCHED FROM ANALYSIS TIME, OBS / ANL DATES =   2026020418
  2026020912
 READ_TCPS:  IGNORE TC_VITALS ENTRY #            3    
 READ_TCPS:  MISMATCHED FROM ANALYSIS TIME, OBS / ANL DATES =   2026020618
  2026020912
 READ_TCPS:  IGNORE TC_VITALS ENTRY #            4    
 READ_TCPS:  MISMATCHED FROM ANALYSIS TIME, OBS / ANL DATES =   2026020818
  2026020912
 READ_TCPS:  NUMBER OF OBS READ IN =            0    
 READ_TCPS: # out of domain =           0

The case was a C1152C384 warm restart with 2 ensemble members. The GFS forecast was only run out 24 hours to spare resources. The experiment directory is /lfs/h2/emc/ptmp/david.huber/rt_tropcy_qc/EXP/rt_tropcy. The COM directory is /lfs/h2/emc/ptmp/david.huber/rt_tropcy_qc/COM/rt_tropcy.

@DavidHuber-NOAA
Copy link
Contributor Author

I take it back. The gfs cycle for 2026020918 had one tcvitals entry to process in the gfs_anal job:

 READ_TCPS:  IANLDATE =   2026020918
 READ_TCPS:  IGNORE TC_VITALS ENTRY #            2    
 READ_TCPS:  MISMATCHED FROM ANALYSIS TIME, OBS / ANL DATES =   2026013018
  2026020918
 READ_TCPS:  IGNORE TC_VITALS ENTRY #            3    
 READ_TCPS:  MISMATCHED FROM ANALYSIS TIME, OBS / ANL DATES =   2026020418
  2026020918
 READ_TCPS:  IGNORE TC_VITALS ENTRY #            4    
 READ_TCPS:  MISMATCHED FROM ANALYSIS TIME, OBS / ANL DATES =   2026020618
  2026020918
 READ_TCPS:  NUMBER OF OBS READ IN =            1    
 READ_TCPS: # out of domain =           0
...
    J contribution                 1             2             3             4             5             6             7
excess   moisture       7.260864E-01  7.002501E-01  6.770496E-01  6.592200E-01  6.324560E-01  6.147118E-01  5.964913E-01
surface pressure        1.745441E+03  1.077682E+03  1.069597E+03  2.402969E+03  1.409566E+03  1.112261E+03  3.861359E+01
temperature             4.230320E+03  9.712872E+03  1.087844E+04  1.216788E+04  1.132972E+04  1.120631E+04  1.383741E+03
wind                    1.142093E+04  3.119187E+04  3.073590E+04  2.453693E+04  2.151107E+04  1.849519E+04  1.521470E+03
moisture                2.947174E+02  5.811586E+02  8.763523E+02  8.557679E+02  7.334591E+02  7.084810E+02  8.325670E+01
sst                     4.848268E+02  5.979244E+02  6.878010E+02  7.183535E+02  6.737587E+02  6.024286E+02  5.430039E+00
ozone                   1.842525E+03  4.360316E+03  4.497397E+03  4.246024E+03  3.355842E+03  3.331850E+03  0.000000E+00
gps bending angle       2.330881E+04  4.256589E+04  4.238695E+04  3.640792E+04  3.631610E+04  1.002257E+04  0.000000E+00
radiance                1.181474E+05  1.884447E+05  1.713500E+05  1.834487E+05  1.830338E+05  9.331926E+04  7.671735E+03
tcp (tropic cyclone)    0.000000E+00  0.000000E+00  0.000000E+00  8.754590E+00  0.000000E+00  0.000000E+00  0.000000E+00
     J term                                     J
nid002317.cactus.wcoss2.ncep.noaa.gov 445: excess   moisture            4.6062652757768801E+00
surface pressure             8.8561292615806615E+03
temperature                  6.0909286730946616E+04
wind                         1.3941336666052919E+05
moisture                     4.1331929534312458E+03
sst                          3.7705231468313032E+03
ozone                        2.1633953748924570E+04
gps bending angle            1.9100823178375067E+05
radiance                     9.4541569846256124E+05
tcp (tropic cyclone)         8.7545898035124807E+00
 ----------------------------------------------------- 
 J Global                    1.3751537436036346E+06

@DavidHuber-NOAA
Copy link
Contributor Author

DavidHuber-NOAA commented Feb 10, 2026

On closer examination, and in comparison with the GFS v16 tcvitals files, I think the files being generated are not correct. This will likely need some digging. The operational /lfs/h1/ops/prod/com/gfs/v16.3/gfs.20260209/18/atmos/gfs.t18z.syndata.tcvitals.tm00 contains

JTWC 21S GEZANI    20260209 1800 181S 0529E 265 042 0987 1012 0398 36 027 0093 0130 0093 0093 D
JTWC 20S MITCHELL  20260209 1200 253S 1138E 180 057 0991 1004 0315 23 046 0130 0121 0111 0111 M

while this test's /lfs/h2/emc/ptmp/david.huber/rt_tropcy_qc/COM/rt_tropcy/gfs.20260209/18/obs/gfs.t18z.syndata.tcvitals.tm00 contains

JTWC 21S GEZANI    20260209 1800 181S 0529E 265 042 0987 1012 0398 36 027 0093 0130 0093 0093 D
JTWC 19S FYTIA     20260130 1800 155S 0439E 085 026 0963 1007 0324 48 022 0148 0130 0111 0148 D
JTWC 02W PENHA     20260204 1800 083N 1304E 260 052 1001 1008 0296 15 111 -999 -999 -999 -999 S
JTWC 20S MITCHELL  20260206 1800 189S 1190E 250 052 0989 1002 0259 26 037 0111 0130 0130 0111 D

It seems that the dates are not correct for all but storm GEZANI. It's not clear to me at this time where these are derived. I am going to convert this PR back to draft while I investigate.

@DavidHuber-NOAA DavidHuber-NOAA marked this pull request as draft February 10, 2026 18:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Missing tcvitals file in V17

1 participant