Add new feature that can plot global Bias and RMSE

[mingchen-NOAA]

Pull Request Summary

This PR adds new features for plotting global bias and RMSE. It also improves the efficiency of colored scatter plots for visualizing sample density.

Description

The python script, pvalstats.py is modified as following:

Global Bias and RMSE plot capability:
Developed a new class, GlobalSkillMap to process, calculate, and plot.

• lat/lon binning (centers)
• bias = mean(model - obs)
• rmse = sqrt(mean((model - obs)^2))
• min_count masking
• pcolormesh map with fixed vmin/vmax (no cartopy)

Colored scatter plot:
Added five downsampling methods for different needs.

• 'uniform': Evenly spaced sampling (default, original method)
• 'uniform_max': Selects the largest observation value in each evenly spaced sector.
• 'random': Random sampling of point
• 'datadensity': Sampling inversely proportional to data density
• 'quantile': Sampling equally for quantile bins

Example plotting scripts work on Ursa:
eval.py
evalsumconfig.json

Issue(s) addressed

fixes #79
fixes #80

Commit Message

Add new features that can plot global Bias and RMSE

Check list

• Branch is up to date with the authoritative repository (NOAA-EMC) develop branch.

Testing

• How were these changes tested?
  Tested using retrotests data.

[JessicaMeixner-NOAA]

@mingchen-NOAA Thank you so much for making a PR and addressing these two updates!

One quick question I have:

Should the creation of the global plots be in ‎ww3tools/pvalstats.py or should it be in ww3tools/??? .py ?

[sbanihash]

Thanks Ming for this PR. Could you also provide test instructions as well so that I could test and merge the PR faster?

Saeideh

[mingchen-NOAA]

@mingchen-NOAA Thank you so much for making a PR and addressing these two updates!

One quick question I have:

Should the creation of the global plots be in ‎ww3tools/pvalstats.py or should it be in ww3tools/??? .py ?

@JessicaMeixner-NOAA I would suggest placing it in pvalstats.py. I don’t see any drawbacks to doing so, and it can be used in the similar way as the QQ and Taylor plots. Do you have any concerns?

[mingchen-NOAA]

Thanks Ming for this PR. Could you also provide test instructions as well so that I could test and merge the PR faster?

Saeideh

@sbanihash I have a script eval.py (configuration file evalsumconfig.json) on Ursa
/scratch4/NCEPDEV/marine/Ming.Chen/wave_eval/HR_eval/hr-eval/

You can check and run it (I tested it using CRYOSAT2, you can use other satellites).

[JessicaMeixner-NOAA]

I don't usually use the taylor or QQ plots, so I was thinking it was out of place in this script, but it clearly is not. Thanks for explaining.

[JessicaMeixner-NOAA]

@mingchen-NOAA - Should your script eval.py be added to say the hr-eval folder?

[mingchen-NOAA]

@JessicaMeixner-NOAA My eval.py and evalsumconfig.json is updated for Ursa. We can add a PR/folder including all evaluation scripts needed for Ursa.

My longer-term goal is to remove all hard-coded directories and machine-specific settings so the evaluation can run on any machine without modifying variables across multiple scripts. This effort is currently considered a lower priority.

[sbanihash]

@mingchen-NOAA I will start working on this PR today. Do you by any chance have a sample path and plot for me to compare my test with?

[mingchen-NOAA]

@mingchen-NOAA I will start working on this PR today. Do you by any chance have a sample path and plot for me to compare my test with?

@sbanihash I have plots for GFSv16 in /scratch4/NCEPDEV/marine/Ming.Chen/wave_eval/processsatdata_ursa/outeval/winter/GFSv16

Retrotests in /scratch4/NCEPDEV/marine/Ming.Chen/wave_eval/processsatdata_ursa/outeval/winter/Retrotests

I got these output using eval.py in /scratch4/NCEPDEV/marine/Ming.Chen/wave_eval/HR_eval/hr-eval