Skip to content

Commit

Permalink
geospatial analysis use auto_template
Browse files Browse the repository at this point in the history
  • Loading branch information
tobin-ford committed Nov 1, 2024
1 parent 0ef212c commit a4630af
Show file tree
Hide file tree
Showing 9 changed files with 115 additions and 14 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
pvdeg.geospatial.plot\_sparse\_analysis\_land
=============================================

.. currentmodule:: pvdeg.geospatial

.. autofunction:: plot_sparse_analysis_land
8 changes: 8 additions & 0 deletions docs/source/_autosummary/pvdeg.geospatial.rst
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ pvdeg.geospatial
pvdeg.geospatial.plot_Europe
pvdeg.geospatial.plot_USA
pvdeg.geospatial.plot_sparse_analysis
pvdeg.geospatial.plot_sparse_analysis_land
pvdeg.geospatial.start_dask
pvdeg.geospatial.template_parameters
pvdeg.geospatial.zero_template
Expand Down Expand Up @@ -157,6 +158,13 @@ pvdeg.geospatial
.. minigallery:: pvdeg.geospatial.plot_sparse_analysis
:add-heading:

.. autofunction:: plot_sparse_analysis_land

.. _sphx_glr_backref_pvdeg.geospatial.plot_sparse_analysis_land:

.. minigallery:: pvdeg.geospatial.plot_sparse_analysis_land
:add-heading:

.. autofunction:: start_dask

.. _sphx_glr_backref_pvdeg.geospatial.start_dask:
Expand Down
6 changes: 6 additions & 0 deletions docs/source/_autosummary/pvdeg.utilities.linear_normalize.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
pvdeg.utilities.linear\_normalize
=================================

.. currentmodule:: pvdeg.utilities

.. autofunction:: linear_normalize
6 changes: 6 additions & 0 deletions docs/source/_autosummary/pvdeg.utilities.merge_sparse.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
pvdeg.utilities.merge\_sparse
=============================

.. currentmodule:: pvdeg.utilities

.. autofunction:: merge_sparse
16 changes: 16 additions & 0 deletions docs/source/_autosummary/pvdeg.utilities.rst
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ pvdeg.utilities
pvdeg.utilities.get_kinetics
pvdeg.utilities.get_state_bbox
pvdeg.utilities.gid_downsampling
pvdeg.utilities.linear_normalize
pvdeg.utilities.merge_sparse
pvdeg.utilities.meta_as_dict
pvdeg.utilities.new_id
pvdeg.utilities.nrel_kestrel_check
Expand Down Expand Up @@ -100,6 +102,20 @@ pvdeg.utilities
.. minigallery:: pvdeg.utilities.gid_downsampling
:add-heading:

.. autofunction:: linear_normalize

.. _sphx_glr_backref_pvdeg.utilities.linear_normalize:

.. minigallery:: pvdeg.utilities.linear_normalize
:add-heading:

.. autofunction:: merge_sparse

.. _sphx_glr_backref_pvdeg.utilities.merge_sparse:

.. minigallery:: pvdeg.utilities.merge_sparse
:add-heading:

.. autofunction:: meta_as_dict

.. _sphx_glr_backref_pvdeg.utilities.meta_as_dict:
Expand Down
2 changes: 2 additions & 0 deletions docs/source/whatsnew/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ What's New
PVDegradationTools (pvdeg) change log:


.. include:: releases/v0.4.3.rst
.. include:: releases/v0.4.2.rst
.. include:: releases/v0.4.1.rst
.. include:: releases/v0.4.0.rst
.. include:: releases/v0.3.3.rst
Expand Down
4 changes: 2 additions & 2 deletions docs/source/whatsnew/releases/v0.4.2.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@ v0.4.2 (2024-09-13)

Bug Fixes
---------
* Remove duplicate gid's from `pvdeg.geospatial.elevation_stochastic_downselection`
* Remove duplicate gid's from ``pvdeg.geospatial.elevation_stochastic_downselection``

Tests
-----
* Added a test for Xmin in `test_standards.py` and removed dependency on pvgis.
* Added a test for Xmin in ``test_standards.py`` and removed dependency on pvgis.

Contributors
~~~~~~~~~~~~
Expand Down
10 changes: 10 additions & 0 deletions docs/source/whatsnew/releases/v0.4.3.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
v0.4.3 (2024-11-1)
=======================

Enhancements
-------------
``pvdeg.geospatial.analysis`` implements autotemplating. No need to specify a template for common ``pvdeg`` functions during analysis. Manually creating and providing templates is still an option. Docstrings updated with examples.

Contributors
~~~~~~~~~~~~
* Tobin Ford (:ghuser:`tobin-ford`)
71 changes: 59 additions & 12 deletions pvdeg/geospatial.py
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,12 @@ def calc_block(weather_ds_block, future_meta_df, func, func_kwargs):

def analysis(weather_ds, meta_df, func, template=None, **func_kwargs):
"""
Applies a function to each gid of a weather dataset.
Applies a function to each gid of a weather dataset. `analysis` will attempt to create a template using `geospatial.auto_template`.
If this process fails you will have to provide a geospatial template to the template argument.
ValueError: <function-name> cannot be autotemplated. create a template manually with `geospatial.output_template`
Parameters
----------
Expand All @@ -212,8 +217,10 @@ def analysis(weather_ds, meta_df, func, template=None, **func_kwargs):
"""

if template is None:
param = template_parameters(func)
template = output_template(weather_ds, **param)
template = auto_template(
func=func,
ds_gids=weather_ds
)

# future_meta_df = client.scatter(meta_df)
kwargs = {"func": func, "future_meta_df": meta_df, "func_kwargs": func_kwargs}
Expand Down Expand Up @@ -241,14 +248,43 @@ def analysis(weather_ds, meta_df, func, template=None, **func_kwargs):


def output_template(
ds_gids, shapes, attrs=dict(), global_attrs=dict(), add_dims=dict()
ds_gids: xr.Dataset,
shapes: dict,
attrs=dict(),
global_attrs=dict(),
add_dims=dict()
):
"""
Generates a xarray template for output data. Output variables and
associated dimensions need to be specified via the shapes dictionary.
The dimension length are derived from the input data. Additonal output
dimensions can be defined with the add_dims argument.
Examples
--------
Providing the shapes dictionary can be confusing. Here is what the `shapes` dictionary should look like for `pvdeg.standards.standoff`.
Refer to the docstring, the function will have one result per location so the only dimension for each return value is "gid", a geospatial ID number.
.. code-block:: python
shapes = {
"x": ("gid",),
"T98_inf": ("gid",),
"T98_0": ("gid",),
}
**Note: The dimensions are stored in a tuple, this this why all of the parenthesis have commas after the single string, otherwise python will interpret the value as a string.**
This is what the shapes dictinoary should look like for `pvdeg.humidity.module`. Refering to the docstring,
we can see that the function will return a timeseries result for each location. This means we need dimensions of "gid" and "time".
.. code-block:: python
shapes = {
"RH_surface_outside": ("gid", "time"),
"RH_front_encap": ("gid", "time"),
"RH_back_encap": ("gid", "time"),
"RH_backsheet": ("gid", "time"),
}
Parameters
----------
ds_gids : xarray.Dataset
Expand Down Expand Up @@ -290,9 +326,8 @@ def output_template(
return output_template


# we should be able to get rid of this with the new autotemplating function and decorator
# this is helpful for users so we should move it to a section in the documenation,
# discuss with group
# This has been replaced with pvdeg.geospatial.auto_templates inside of pvdeg.geospatial.analysis.
# it is here for completeness. it can be removed.
def template_parameters(func):
"""
Output parameters for xarray template.
Expand Down Expand Up @@ -417,11 +452,10 @@ def auto_template(func: Callable, ds_gids: xr.Dataset) -> xr.Dataset:
Only works on functions that have the `numeric_or_timeseries` and `shape_names` attributes.
These attributes are assigned at function definition with the `@geospatial_quick_shape` decorator.
Otherwise you will have to create your own template.
Don't worry, this is easy. See the Geospatial Templates Notebook
for more information.
Otherwise you will have to create your own template using `geospatial.output_template`.
See the Geospatial Templates Notebook for more information.
examples:
Examples
---------
the function returns a numeric value
Expand All @@ -430,11 +464,24 @@ def auto_template(func: Callable, ds_gids: xr.Dataset) -> xr.Dataset:
the function returns a timeseries result
>>> pvdeg.module.humidity
counter example:
Counter example:
----------------
the function could either return a single numeric or a series based on changed in
the input. Because it does not have a known result shape we cannot determine the
attributes required for autotemplating ahead of time.
Parameters
----------
func: callable
function to create template from. This will raise an error if the function was not declared with the `@geospatial_quick_shape` decorator.
ds_gids : xarray.Dataset
Dataset containing the gids and their associated dimensions. (geospatial weather dataset)
Dataset should already be chunked.
Returns
-------
output_template : xarray.Dataset
Template for output data.
"""

if not (hasattr(func, "numeric_or_timeseries") and hasattr(func, "shape_names")):
Expand Down

0 comments on commit a4630af

Please sign in to comment.