Output switches for output volumes of iHAMOCC

[jmaerz]

Hi @JorgSchwinger and @TomasTorsvik,
since we discussed this briefly after lunch, I started to implement at least the required switch to set different iHAMOCC output options. As is, it works for the thus far example variable SRF_PHOSPH while it doesn't change any default behavior. This is just for demo and need to be concluded via introducing the settings throughout namelist_definition_blom.xml (in another PR, ones defined the output settings).

If you would opt for a different switch name, let me know. Also about the different options (i.e. adding a default option).

[JorgSchwinger]

Thanks for doing this so quickly. Do you think it would be feasible to write a script to roll this out to all variables?

[JorgSchwinger]

I would suggest a more self-explaining name, e.g. HAMOCC_OUTPUT_VOL(UME) or HAMOCC_OUTPUT_SIZE

[jmaerz]

Renamed to HAMOCC_OUTPUT_SIZE

[JorgSchwinger]

For daily output it needs to be 4. Or more precisely, whenever more than one time-slice is written into one file, one cannot use 2. The compressed format with offset and scale-factor does not work properly in this case.

[jmaerz]

Can you re-check? The newly pushed commit should have the right output, right?

[jmaerz]

Hi @JorgSchwinger , I am just about investigating how to best do this via a xml-parse script... - don't now, how deep the rabbit hole I want to go, though. I'll keep you updated.

[jmaerz]

Ok, I technically have a parser routine that allows to set those values for all outputs. However, I wonder, whether this is really the way to go. I'll share the python script that I have (requires minor manual modifications afterward - didn't get some white lines correct...).

[jmaerz]

Hi @JorgSchwinger and @TomasTorsvik , how to best proceed? - I feel not being the person who should decide about the volumes for the individual output variables (only have some idea about certain ones). I would propose to declare one default flag in addition to small, medium, large, where we enter the current values - so no changes to the current version. This I would merge. Then, if there is time, to go through the variables, setting just the output values manually (i.e. the full structure can be generated via the script, I sent around).

[TomasTorsvik]

Looks good to me. I think a default option with the current values would be useful.

[jmaerz]

I would have a routine ready that basically performs such changes for any iHAMOCC variable (left new, right old):

Note to myself: Requires default as option in the compsets

[JorgSchwinger]

Hi @JorgSchwinger and @TomasTorsvik , how to best proceed? - I feel not being the person who should decide about the volumes for the individual output variables (only have some idea about certain ones). I would propose to declare one default flag in addition to small, medium, large, where we enter the current values - so no changes to the current version. This I would merge. Then, if there is time, to go through the variables, setting just the output values manually (i.e. the full structure can be generated via the script, I sent around).

Why not define one of small/medium/large as the default? I would not use the current values as default, but rather something with less output. Given that storage is scarce, I would think it's better that the user needs to actively increase the storage volume if this is desired.

I will come up with a suggestion as to what the categories should contain, but I'll have to do this later (next week)

[JorgSchwinger]

I'm wondering if this is necessary - the parser will have a logic of taking the last match that is found? At least otherwise, the existing is_test and empty_hist logic wouldn't work either.

[jmaerz]

Hi @JorgSchwinger , I am happy, when you flee out the desired output volumes - I just see that there are cases (e.g. CO2 fluxes come to my mind) where the current value is quite different from small/medium/large as compared to the other variables. A default would allow to keep the output as is and having standardized other output volumes. But up to you - it's just needed to manually go through, if there are special wishes/needs - it was just a thought to roll this out now and enabling an easy comparison, when manually adjusting the stuff to your/our needs.

[jmaerz]

Hi @TomasTorsvik , @JorgSchwinger and @matsbn , I updated the namelist_definition_blom.xml file according to the information gathered in #498 and provide two switches - one for BLOM, one for HAMOCC. Untested thus far. I wonder, if we want to keep the small, medium and large in addition (for both, BLOM and HAMOCC)?
Further, we still need to add the shell scripts for making the switches available to NorESM as discussed in #498 and NorESMhub/NorESM#654

[JorgSchwinger]

Hi @jmaerz , thanks a lot for preparing this! It looks good to me (but of course I didn't go through the whole namelist-creation file...). Since we and others want to use this soon (now), I would suggest to briefly test if the setting works and produces the expected results and then go ahead with this. Other and more options will be useful, but maybe can be added later?

[TomasTorsvik]

@jmaerz - Thanks for preparing this!

[jmaerz]

In which NorESM version should this be tested?

[TomasTorsvik]

In which NorESM version should this be tested?

I can test this for noresm3_0_alpha01, N1850 compset. Can you test it for noresm2.3? I suppose 1 year run should be sufficient.

[jmaerz]

Hi @TomasTorsvik , thanks, yes, I can give it a try in NorESM2.3.

[jmaerz]

Just to document, what is needed to make use of the reduced output size switches:

${SRCROOT}/cime/scripts/create_newcase --case ${CASEDIR} --res ${GRID} --mach ${MACHINE} --compset ${COMPSET} --project ${PROJECT} --run-unsupported --user-mods-dir $SRCROOT/components/blom/cime_config/usermods_dirs/reduced_out_devsim/

Investigating ./CaseDocs/ocn_in for NorESM2.3 alpha03 after setting up the run, this looks good - I'll perform a test run.

[jmaerz]

Ok, the functionality works as expected, but I do see that some bits were missed by the script - I'll try to fix this.

[TomasTorsvik]

@jmaerz - It also works for noresm3_0_alpha01 (did not complete the full year run yet). At least I'm seeing reduced output in monthly files. So it should be safe to merge this PR when you feel it is ready.

[jmaerz]

@TomasTorsvik , thanks for letting me know. For the last commit, I updated the script and checked the written namelist_definition_blom.xml file. I missed to run the simulation with the extended N-cycle, which I will do after merging the PR (if changes are needed for the Sankey-diag, I will do this in a separate PR).