Add metadata for BIOS ancillaries by SeanBryan51 · Pull Request #3 · CABLE-LSM/gists

SeanBryan51 · 2024-10-21T01:30:13Z

This change adds metadata to the BIOS gridinfo file as per discussions in CABLE-LSM/CABLE#406.

SeanBryan51 · 2024-10-21T06:11:19Z

Here is a before and after diff of the header file changes (ignoring history logs) (NOW OUTDATED):

diff -u <(ncdump -h gridinfo_out_main.nc | sed '/Mon Oct 21/d') <(ncdump -h gridinfo_out_add-bios-provenance-information.nc | sed '/Mon Oct 21/d')

--- /dev/fd/63	2024-10-21 17:08:37.446300304 +1100
+++ /dev/fd/62	2024-10-21 17:08:37.446300304 +1100
@@ -1,4 +1,4 @@
-netcdf gridinfo_out_main {
+netcdf gridinfo_out_add-bios-provenance-information {
 dimensions:
 	latitude = 670 ;
 	longitude = 813 ;
@@ -147,57 +147,91 @@
 		crs:spatial_ref = "GEOGCS[\"GDA94\",DATUM[\"Geocentric_Datum_of_Australia_1994\",SPHEROID[\"GRS 1980\",6378137,298.257222101,AUTHORITY[\"EPSG\",\"7019\"]],AUTHORITY[\"EPSG\",\"6283\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.0174532925199433,AUTHORITY[\"EPSG\",\"9122\"]],AXIS[\"Latitude\",NORTH],AXIS[\"Longitude\",EAST],AUTHORITY[\"EPSG\",\"4283\"]]" ;
 		crs:GeoTransform = "112.925 0.05 0 -10.075 0 -0.05 " ;
 	float silt(latitude, longitude) ;
-		silt:long_name = "GDAL Band Number 1" ;
+		silt:long_name = "UM soil texture - silt fraction" ;
 		silt:_FillValue = -9999.f ;
 		silt:grid_mapping = "crs" ;
+		silt:units = "1" ;
+		silt:comment = "Data provided from the Atlas of Australian Soils (digital)" ;
+		silt:references = "https://doi.org/10.25919/5df03e991672d and https://publications.csiro.au/publications/publication/PIlegacy:889" ;
 	float clay(latitude, longitude) ;
-		clay:long_name = "GDAL Band Number 1" ;
+		clay:long_name = "UM soil texture - clay fraction" ;
 		clay:_FillValue = -9999.f ;
 		clay:grid_mapping = "crs" ;
+		clay:units = "1" ;
+		clay:comment = "Data provided from the Atlas of Australian Soils (digital)" ;
+		clay:references = "https://doi.org/10.25919/5df03e991672d and https://publications.csiro.au/publications/publication/PIlegacy:889" ;
 	float sand(latitude, longitude) ;
 		sand:_FillValue = -9999.f ;
 		sand:grid_mapping = "crs" ;
-		sand:long_name = "GDAL Band Number 1" ;
+		sand:long_name = "UM soil texture - sand fraction" ;
+		sand:units = "1" ;
+		sand:comment = "Data provided from the Atlas of Australian Soils (digital)" ;
+		sand:references = "https://doi.org/10.25919/5df03e991672d and https://publications.csiro.au/publications/publication/PIlegacy:889" ;
 	float css(latitude, longitude) ;
-		css:long_name = "GDAL Band Number 1" ;
+		css:long_name = "Soil specific heat capacity" ;
 		css:_FillValue = -9999.f ;
 		css:grid_mapping = "crs" ;
+		css:units = "J/kg/K" ;
+		css:comment = "Data provided from the Atlas of Australian Soils (digital)" ;
+		css:references = "https://doi.org/10.25919/5df03e991672d and https://publications.csiro.au/publications/publication/PIlegacy:889" ;
 	float sfc(latitude, longitude) ;
-		sfc:long_name = "GDAL Band Number 1" ;
+		sfc:long_name = "VOL SMC AT CRIT PT" ;
 		sfc:_FillValue = -9999.f ;
 		sfc:grid_mapping = "crs" ;
+		sfc:units = "m3 m-3" ;
+		sfc:comment = "Data provided from the Atlas of Australian Soils (digital)" ;
+		sfc:references = "https://doi.org/10.25919/5df03e991672d and https://publications.csiro.au/publications/publication/PIlegacy:889" ;
 	float rhosoil(latitude, longitude) ;
-		rhosoil:long_name = "GDAL Band Number 1" ;
+		rhosoil:long_name = "SOIL BULK DENSITY" ;
 		rhosoil:_FillValue = -9999.f ;
 		rhosoil:grid_mapping = "crs" ;
+		rhosoil:units = "kg/m^3" ;
+		rhosoil:comment = "Data provided from the Atlas of Australian Soils (digital)" ;
+		rhosoil:references = "https://doi.org/10.25919/5df03e991672d and https://publications.csiro.au/publications/publication/PIlegacy:889" ;
 	float bch(latitude, longitude) ;
 		bch:_FillValue = -9999.f ;
 		bch:grid_mapping = "crs" ;
-		bch:long_name = "GDAL Band Number 1" ;
+		bch:long_name = "CLAPP-HORNBERGER B COEFFICIENT" ;
+		bch:units = "1" ;
+		bch:comment = "Data provided from the Atlas of Australian Soils (digital)" ;
+		bch:references = "https://doi.org/10.25919/5df03e991672d and https://publications.csiro.au/publications/publication/PIlegacy:889" ;
 	float hyds(latitude, longitude) ;
 		hyds:_FillValue = -9999.f ;
 		hyds:grid_mapping = "crs" ;
-		hyds:long_name = "GDAL Band Number 1" ;
+		hyds:long_name = "SAT SOIL CONDUCTIVITY" ;
+		hyds:units = "m/s" ;
+		hyds:comment = "Data provided from the Atlas of Australian Soils (digital)" ;
+		hyds:references = "https://doi.org/10.25919/5df03e991672d and https://publications.csiro.au/publications/publication/PIlegacy:889" ;
 	float ssat(latitude, longitude) ;
 		ssat:_FillValue = -9999.f ;
 		ssat:grid_mapping = "crs" ;
-		ssat:long_name = "GDAL Band Number 1" ;
+		ssat:long_name = "VOL SMC AT SATURATION" ;
+		ssat:units = "1" ;
+		ssat:comment = "Data provided from the Atlas of Australian Soils (digital)" ;
+		ssat:references = "https://doi.org/10.25919/5df03e991672d and https://publications.csiro.au/publications/publication/PIlegacy:889" ;
 	float swilt(latitude, longitude) ;
 		swilt:_FillValue = -9999.f ;
 		swilt:grid_mapping = "crs" ;
-		swilt:long_name = "GDAL Band Number 1" ;
+		swilt:long_name = "VOL SMC AT WILTING" ;
+		swilt:units = "1" ;
+		swilt:comment = "Data provided from the Atlas of Australian Soils (digital)" ;
+		swilt:references = "https://doi.org/10.25919/5df03e991672d and https://publications.csiro.au/publications/publication/PIlegacy:889" ;
 	float sucs(latitude, longitude) ;
 		sucs:_FillValue = -9999.f ;
 		sucs:grid_mapping = "crs" ;
-		sucs:long_name = "GDAL Band Number 1" ;
+		sucs:long_name = "SATURATED SOIL WATER SUCTION" ;
+		sucs:units = "m" ;
+		sucs:comment = "Data provided from the Atlas of Australian Soils (digital)" ;
+		sucs:references = "https://doi.org/10.25919/5df03e991672d and https://publications.csiro.au/publications/publication/PIlegacy:889" ;
 	int mvg(latitude, longitude) ;
 		mvg:_FillValue = -9999 ;
 		mvg:grid_mapping = "crs" ;
-		mvg:long_name = "GDAL Band Number 1" ;
+		mvg:long_name = "NVIS major vegetation group (v5.0)" ;
 	float c4frac(latitude, longitude) ;
-		c4frac:long_name = "GDAL Band Number 1" ;
+		c4frac:long_name = "Fractional cover of C4 grass" ;
 		c4frac:_FillValue = -9999.f ;
 		c4frac:grid_mapping = "crs" ;
+		c4frac:units = "1" ;
 
 // global attributes:
 		:title = "CABLE gridinfo file reconciled with CRU forcing" ;
@@ -210,7 +244,7 @@
 			"/home/189/sb8430/gists/regrid_aus_05x05_to_005x005.sh: Clip to Australian region and subsample from 0.5 degrees resolution to 0.05 degrees using nearest neighbour interpolation. 21.10.2024\n",
 			"One iveg and patchfrac dimension added for savanna-tiling. L. Nieradzik 11/2015" ;
 		:map_file = "map_05x05_to_005x005_neareststod.nc" ;
-		:input_file = "gridinfo_out_main.nc" ;
+		:input_file = "gridinfo_out_add-bios-provenance-information.nc" ;
 			"Wed May 31 00:52:23 2023: ncrename -v Band1,c4_grass_frac_cov c4_grass_frac_cov.nc\n",
 			"Wed May 31 00:31:01 2023: GDAL CreateCopy( c4_grass_frac_cov.nc, ... )\n",
 			"Wed May 31 00:52:32 2023: ncrename -v Band1,nvis5pre1750grp nvis5pre1750grp.nc\n",

AlisonBennett · 2024-10-21T23:31:43Z

A few notes from me:

Could we be consistent in whether the long_name is capitalised or non-capitalised? Clay, silt, sand, specific heat capacity, NVIS major vegetation group, and C4 grass fractional cover are all in sentance-case with the remainder in upper case.
Also the unit convention is inconsistent: I think unit of "VOL SMC AT CRIT PT" should changed from "m3 m-3" to "m^3/m^3" (if I've interpreted that correctly)
Update NVIS major vegetation group (v5.0) to include:
- mvg:comment Data provided from National Vegetation Information System (NVIS)
- mvg:references https://www.dcceew.gov.au/environment/land/native-vegetation/national-vegetation-information-system/data-products#mvg5
Can we find the reference for C4 grass fractional cover? My understanding is that these are developed from the approach in [Donohue et al 2009](https:// doi. org/ 10. 1111/j. 1365- 2486. 2008. 01746. x) but it would be nice to confirm that before putting it in the code. Perhaps @har917 can help with this?

SeanBryan51 · 2024-10-22T00:49:51Z

@AlisonBennett Thanks for taking a look. I've set the sfc units to 1 so that it is consistent with the other dimensionless variables. The long_name descriptions are now in upper case. Waiting on @har917 to confirm the references for c4frac.

abhaasgoyal · 2025-01-22T04:03:54Z

I made the changes in the references for c4frac as mentioned here. Other proposed changes seem to be implemented. @har917 can you look whether it is ready to be merged/any changes required?

--- /dev/fd/63  2025-01-22 14:59:29.404801158 +1100
+++ /dev/fd/62  2025-01-22 14:59:29.404801158 +1100
@@ -1,4 +1,4 @@
-netcdf gridinfo_CSIRO_CRU005x005_4tiles {
+netcdf gridinfo_CSIRO_CRU005x005_4tiles_v2_with_provenance {
 dimensions:
        latitude = 670 ;
        longitude = 813 ;
@@ -147,214 +147,226 @@
                crs:spatial_ref = "GEOGCS[\"GDA94\",DATUM[\"Geocentric_Datum_of_Australia_1994\",SPHEROID[\"GRS 1980\",6378137,298.257222101,AUTHORITY[\"EPSG\",\"7019\"]],AUTHORITY[\"EPSG\",\"6283\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.0174532925199433,AUTHORITY[\"EPSG\",\"9122\"]],AXIS[\"Latitude\",NORTH],AXIS[\"Longitude\",EAST],AUTHORITY[\"EPSG\",\"4283\"]]" ;
                crs:GeoTransform = "112.925 0.05 0 -10.075 0 -0.05 " ;
        float silt(latitude, longitude) ;
-               silt:long_name = "GDAL Band Number 1" ;
+               silt:long_name = "UM SOIL TEXTURE - SILT FRACTION" ;
                silt:_FillValue = -9999.f ;
                silt:grid_mapping = "crs" ;
+               silt:units = "1" ;
+               silt:comment = "Data provided from the Atlas of Australian Soils (digital)" ;
+               silt:references = "https://doi.org/10.25919/5df03e991672d and https://publications.csiro.au/publications/publication/PIlegacy:889" ;
        float clay(latitude, longitude) ;
-               clay:long_name = "GDAL Band Number 1" ;
+               clay:long_name = "UM SOIL TEXTURE - CLAY FRACTION" ;
                clay:_FillValue = -9999.f ;
                clay:grid_mapping = "crs" ;
+               clay:units = "1" ;
+               clay:comment = "Data provided from the Atlas of Australian Soils (digital)" ;
+               clay:references = "https://doi.org/10.25919/5df03e991672d and https://publications.csiro.au/publications/publication/PIlegacy:889" ;
        float sand(latitude, longitude) ;
                sand:_FillValue = -9999.f ;
                sand:grid_mapping = "crs" ;
-               sand:long_name = "GDAL Band Number 1" ;
+               sand:long_name = "UM SOIL TEXTURE - SAND FRACTION" ;
+               sand:units = "1" ;
+               sand:comment = "Data provided from the Atlas of Australian Soils (digital)" ;
+               sand:references = "https://doi.org/10.25919/5df03e991672d and https://publications.csiro.au/publications/publication/PIlegacy:889" ;
        float css(latitude, longitude) ;
-               css:long_name = "GDAL Band Number 1" ;
+               css:long_name = "SOIL SPECIFIC HEAT CAPACITY" ;
                css:_FillValue = -9999.f ;
                css:grid_mapping = "crs" ;
+               css:units = "J/kg/K" ;
+               css:comment = "Data provided from the Atlas of Australian Soils (digital)" ;
+               css:references = "https://doi.org/10.25919/5df03e991672d and https://publications.csiro.au/publications/publication/PIlegacy:889" ;
        float sfc(latitude, longitude) ;
-               sfc:long_name = "GDAL Band Number 1" ;
+               sfc:long_name = "VOL SMC AT CRIT PT" ;
                sfc:_FillValue = -9999.f ;
                sfc:grid_mapping = "crs" ;
+               sfc:units = "1" ;
+               sfc:comment = "Data provided from the Atlas of Australian Soils (digital)" ;
+               sfc:references = "https://doi.org/10.25919/5df03e991672d and https://publications.csiro.au/publications/publication/PIlegacy:889" ;
        float rhosoil(latitude, longitude) ;
-               rhosoil:long_name = "GDAL Band Number 1" ;
+               rhosoil:long_name = "SOIL BULK DENSITY" ;
                rhosoil:_FillValue = -9999.f ;
                rhosoil:grid_mapping = "crs" ;
+               rhosoil:units = "kg/m^3" ;
+               rhosoil:comment = "Data provided from the Atlas of Australian Soils (digital)" ;
+               rhosoil:references = "https://doi.org/10.25919/5df03e991672d and https://publications.csiro.au/publications/publication/PIlegacy:889" ;
        float bch(latitude, longitude) ;
                bch:_FillValue = -9999.f ;
                bch:grid_mapping = "crs" ;
-               bch:long_name = "GDAL Band Number 1" ;
+               bch:long_name = "CLAPP-HORNBERGER B COEFFICIENT" ;
+               bch:units = "1" ;
+               bch:comment = "Data provided from the Atlas of Australian Soils (digital)" ;
+               bch:references = "https://doi.org/10.25919/5df03e991672d and https://publications.csiro.au/publications/publication/PIlegacy:889" ;
        float hyds(latitude, longitude) ;
                hyds:_FillValue = -9999.f ;
                hyds:grid_mapping = "crs" ;
-               hyds:long_name = "GDAL Band Number 1" ;
+               hyds:long_name = "SAT SOIL CONDUCTIVITY" ;
+               hyds:units = "m/s" ;
+               hyds:comment = "Data provided from the Atlas of Australian Soils (digital)" ;
+               hyds:references = "https://doi.org/10.25919/5df03e991672d and https://publications.csiro.au/publications/publication/PIlegacy:889" ;
        float ssat(latitude, longitude) ;
                ssat:_FillValue = -9999.f ;
                ssat:grid_mapping = "crs" ;
-               ssat:long_name = "GDAL Band Number 1" ;
+               ssat:long_name = "VOL SMC AT SATURATION" ;
+               ssat:units = "1" ;
+               ssat:comment = "Data provided from the Atlas of Australian Soils (digital)" ;
+               ssat:references = "https://doi.org/10.25919/5df03e991672d and https://publications.csiro.au/publications/publication/PIlegacy:889" ;
        float swilt(latitude, longitude) ;
                swilt:_FillValue = -9999.f ;
                swilt:grid_mapping = "crs" ;
-               swilt:long_name = "GDAL Band Number 1" ;
+               swilt:long_name = "VOL SMC AT WILTING" ;
+               swilt:units = "1" ;
+               swilt:comment = "Data provided from the Atlas of Australian Soils (digital)" ;
+               swilt:references = "https://doi.org/10.25919/5df03e991672d and https://publications.csiro.au/publications/publication/PIlegacy:889" ;
        float sucs(latitude, longitude) ;
                sucs:_FillValue = -9999.f ;
                sucs:grid_mapping = "crs" ;
-               sucs:long_name = "GDAL Band Number 1" ;
+               sucs:long_name = "SATURATED SOIL WATER SUCTION" ;
+               sucs:units = "m" ;
+               sucs:comment = "Data provided from the Atlas of Australian Soils (digital)" ;
+               sucs:references = "https://doi.org/10.25919/5df03e991672d and https://publications.csiro.au/publications/publication/PIlegacy:889" ;
        int mvg(latitude, longitude) ;
                mvg:_FillValue = -9999 ;
                mvg:grid_mapping = "crs" ;
-               mvg:long_name = "GDAL Band Number 1" ;
+               mvg:long_name = "NVIS MAJOR VEGETATION GROUP (V5.0)" ;
+               mvg:comment = "Data provided from National Vegetation Information System (NVIS)" ;
+               mvg:references = "https://www.dcceew.gov.au/environment/land/native-vegetation/national-vegetation-information-system/data-products#mvg5" ;
        float c4frac(latitude, longitude) ;
-               c4frac:long_name = "GDAL Band Number 1" ;
+               c4frac:long_name = "FRACTIONAL COVER OF C4 GRASS" ;
                c4frac:_FillValue = -9999.f ;
                c4frac:grid_mapping = "crs" ;
-       int modis_igbp(latitude, longitude) ;
-               modis_igbp:_FillValue = -9999 ;
-               modis_igbp:grid_mapping = "crs" ;
-               modis_igbp:long_name = "GDAL Band Number 1" ;
-       float avgannmax_fapar(latitude, longitude) ;
-               avgannmax_fapar:long_name = "GDAL Band Number 1" ;
-               avgannmax_fapar:_FillValue = -9999.f ;
-               avgannmax_fapar:grid_mapping = "crs" ;
+               c4frac:units = "1" ;
+               c4frac:references = "https://doi.org/10.5194/bg-10-2011-2013 following https://doi.org/10.1007/BF00379569" ;
 
 // global attributes:
                :title = "CABLE gridinfo file reconciled with CRU forcing" ;
                :introduction = "reconciled to CRU forcing landmask; based on interpolated gridinfo 0.5x0.5 interpolated from CABLE 1x1 gridinfo" ;
                :production = "Lei.Cheng@csiro.au & Chris.Lu@csiro.au" ;

# CABLE ## Description Fixes #281 Aside from `climate` parameters, BIOS ancillary data is being read from the gridinfo. They are supposed to be read via a separate BLAZE NetCDF file #392 The original 0.5 degree gridinfo file ([`gridinfo_CSIRO_CRU05x05_4tiles.nc`](https://github.com/CABLE-LSM/bios_testing/blob/b45e0736cfd7fd1de1014319eb551b924542a96c/cable.nml#L4C63-L4C96)) went through 2 stages to make it compatible with reading to ancillary inputs TODO: Once these PRs are merged, point to these specific files 1. Subsampling at 0.05 degrees using script CABLE-LSM/gists#3 2. Add BIOS coastal remapping script for gridinfo to have the same mask for all input parameters. CABLE-LSM/gists#4 ## Type of change New feature ## Testing Run `act9test` and `1000pts-configuration` (TODO: provide links) to `BIOS_ancillary_netcdf` and `CABLE_POP-TRENDY` with `derived_parameters` and `check_parameter_values` being called after `cable_load_bios_params` instead of within `load_parameters`.

SeanBryan51 force-pushed the add-bios-provenance-information branch from 849e94a to 32b6740 Compare October 21, 2024 01:31

Remove TODO

7981fef

SeanBryan51 force-pushed the add-bios-provenance-information branch from 5e4b9c4 to 1d50098 Compare October 21, 2024 01:49

SeanBryan51 changed the title ~~Add bios provenance information~~ Add metadata for BIOS ancillaries Oct 21, 2024

SeanBryan51 force-pushed the add-bios-provenance-information branch from 1d50098 to e52d9dd Compare October 21, 2024 03:33

SeanBryan51 added 2 commits October 21, 2024 15:12

Add comment and references for BIOS soil parameters

eea8da2

Set units and long_name attributes for BIOS variables

813e66f

SeanBryan51 force-pushed the add-bios-provenance-information branch from e52d9dd to 813e66f Compare October 21, 2024 04:12

SeanBryan51 marked this pull request as ready for review October 21, 2024 06:11

SeanBryan51 requested review from AlisonBennett and har917 October 21, 2024 06:12

SeanBryan51 added 3 commits October 22, 2024 11:33

Set sfc units to 1

e8764c0

Use upper case for long_name description

98eeacb

Add comment and references to mvg

855970f

Set c4frac references attribute

aad786c

abhaasgoyal force-pushed the add-bios-provenance-information branch from 5ff5604 to aad786c Compare January 22, 2025 01:38

abhaasgoyal mentioned this pull request Jan 23, 2025

Read standard parameter ancillary data in NetCDF CABLE-LSM/CABLE#324

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add metadata for BIOS ancillaries#3

Add metadata for BIOS ancillaries#3
SeanBryan51 wants to merge 7 commits intomainfrom
add-bios-provenance-information

SeanBryan51 commented Oct 21, 2024 •

edited

Loading

Uh oh!

SeanBryan51 commented Oct 21, 2024 •

edited

Loading

Uh oh!

AlisonBennett commented Oct 21, 2024

Uh oh!

SeanBryan51 commented Oct 22, 2024

Uh oh!

abhaasgoyal commented Jan 22, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

SeanBryan51 commented Oct 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SeanBryan51 commented Oct 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AlisonBennett commented Oct 21, 2024

Uh oh!

SeanBryan51 commented Oct 22, 2024

Uh oh!

abhaasgoyal commented Jan 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

SeanBryan51 commented Oct 21, 2024 •

edited

Loading

SeanBryan51 commented Oct 21, 2024 •

edited

Loading

abhaasgoyal commented Jan 22, 2025 •

edited

Loading