EMCPy V2 by kevindougherty-noaa · Pull Request #159 · NOAA-EMC/emcpy

kevindougherty-noaa · 2025-09-15T21:18:42Z

This pull request merges several recent enhancements, refactors, and documentation improvements into a single cohesive update. It represents a major step toward making EMCPy more robust, extensible, and consistent across use cases.

Highlights

Refactored plotting core for more stable subplot/grid creation and compatibility across Matplotlib versions.
New plot layers added: FillBetween, ErrorBar, Violin, HexBin, and Hist2D.
Improved discrete fields and colorbars with better handling of integer/categorical data.
Hardened APIs across box-and-whisker plots, scatter regression layers, and map adapters.
Centralized tick/formatter logic with datetime-safe handling.
Restructured tests and CI for improved reliability and maintainability.
Documentation overhaul:
- New Explanations section with deeper design notes.
- Expanded gallery with new plot types.
- Cleaner and more consistent narrative throughout.

Breaking Changes / Migration Notes

Plot layer initialization: some classes (e.g., FillBetween) no longer accept certain kwargs (alpha must now be set via attributes after initialization).
Box-and-whisker orientation: unified handling via orientation="vertical"|"horizontal"; the vert flag is no longer passed directly to Matplotlib.
Integer/discrete fields: behavior of integer_field=True has been standardized; results may differ slightly in tick placement or colorbar labeling compared to older versions.
Gallery and docs paths: users linking directly to older gallery examples may need to update references.

Impact

This PR lays the foundation for EMCPy v2 by improving stability, expanding plotting options, and streamlining the developer experience. It enables easier extension of plotting features while maintaining a consistent user interface.

* introduce axstate * fix norms * pycodestyle * test CI change * test again * test * copilots reviews * pycodestyle

… FixedLocator) (#149) * fix tick formatting issue * pycodestyle * copilor reviews

…lpers) and fix single-colorbar/sharex/sharey logic (#151) * update grid position calling * copilot reviews

…d stabilize scatter regression (#152) * address deprecated labels in box and whisker and other fail in Scatter * pycodestyle * copilot reviews

…#150) * update plotting tests * fix errors * pycodestyle * pycodestyle * pycodestyle * minor tweak to scatter linear regression * pycodestyle * handle attribute spellings and public setters correctly * pycodestyle * add fail test for fix in future PR * typo * missing import * missing import * fail safe for another test * update test * update conftest * cleanup * copilot review * fix fixture * missed one

…ing (#153) * Add layer adapter and make changes in renderers * update adapters * revert map adapters * pycodestyle * pycodestyle * pycodestyle * clean up adapters * Address error * add tests for changes made * fix _last_mappable * forgot import * address failing test * fix scatter color issue * another attempt at color error * copilot reviews

…d plotting correctness fixes (#154) * avoid mutable defaults * fix contourf transform on map * clean up invert axis and add tests * tests for invert * strengthen legends * cleanup * add helper in adapters * future proof ticks for GeoAxes * fix deprecated input for boxandwhisker * pycodestyle * Address invert error * pycodestyle * fix test * fix tests again * try again * another test * Attempt invert again * update tests * one more time * remove unnecessary test * quick cleanup * cleanup default arg to avoid mutable defaults * typo * missing import * copilot review

…eprecations (#155) * add more tests to stress test new additions * pynorms * cleanup vmin/vmax norm error * missed import * copilot reviews

…vert) (#157) * add changes to boxplot * copilot reviews

…l `twinx` support (#158) * Add more features * Add tests * add twinx * pycodestyle * update tests * copilot reviews * copilot reviews

* update documentation workflow * removing old files * update new docs main dir * add get-started * add how-to .mds * add explanations .mds * add contributing .mds * update to include gallery * fixes while building html * updates to get running website * clean up site options * add line plot examples to gallery * clean up boxplot features * clean up map errors * more examples * fix all pycodestyle * missed one pycodestyle * fix tests * pycodestyle * pycodestyle * fix class * patch gridded plot * pathc for map_gridded * typos * add new plot types for gallery * update workflow yaml * cleanup * cleanup * rename file * build cleanup * pycodestyle * copilot and Cory reviews

* initial clean up and move .txt to .md files * cleanup readmes for gallery * website builds with zero warnings * avoid cross-call mutations * forgot imports * pynorms * cleans up and prevents norms breaking * pycodestyle * colorbar consistency * add logical validation which raises errors * add more tests * pycodestyle * pynorms * typo * quick fix * remove unsued import * update _norms * pycodestyle * fix * add conditional for strange boundary issue * updates to docs and quick fix * copilot reviews * add import

kevindougherty-noaa · 2025-09-18T15:42:26Z

@EdwardSafford-NOAA @ADCollard Cory has been in the loop on all the PRs merged into this branch, I don't expect you to review 100+ files! I wanted to keep you guys in the loop on a major upgrade to EMCPy and that we will need to update all hashes to anything that is dependent.

Copilot

Pull Request Overview

This pull request represents a major refactor of EMCPy, introducing a new architecture for plot layer rendering, enhanced test coverage, and extensive documentation improvements. The changes modernize the codebase while maintaining backward compatibility for existing APIs.

Key changes include:

Introduced an adapter pattern for plot layer rendering with centralized registry
Added new plot layers: FillBetween, ErrorBar, ViolinPlot, HexBin, and Hist2D
Refactored box-and-whisker plots with improved orientation handling and Matplotlib version compatibility
Enhanced discrete field support with automatic BoundaryNorm generation for integer categories
Comprehensive test suite restructure with improved CI configuration
Documentation overhaul with new gallery examples and explanations sections

Reviewed Changes

Copilot reviewed 95 out of 102 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
src/tests/test_plots.py	Removed legacy test file (729 lines deleted)
src/tests/test_map_plots.py	Removed legacy map test file (235 lines deleted)
src/tests/plotting/test_*.py	Added comprehensive new test suite covering ticks, maps, colorbars, adapters, and layer functionality
src/emcpy/plots/plots.py	Enhanced with new layer classes and improved BoxandWhiskerPlot compatibility
src/emcpy/plots/map_plots.py	Strengthened validation and error handling for map layers
src/emcpy/plots/create_plots.py	Major refactor introducing adapter pattern and enhanced rendering pipeline
src/emcpy/plots/adapters.py	New adapter registry system for plot layer rendering
galleries/	Added extensive example gallery with statistical, map, and gridded plot examples
docs/	Complete documentation restructure with modern Sphinx configuration
pyproject.toml	Modern Python packaging configuration

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-09-18T15:45:16Z

src/emcpy/plots/plots.py

        self.edgecolors = None
        self.label = f'n={np.count_nonzero(~np.isnan(x))}'
        self.do_linear_regression = False
+        # Optional style overrides for the regression line; kept empty by default.


The comment mentions 'kept empty by default' but the code initializes it to an empty dict. The comment should clarify that it's initialized as an empty dictionary to store optional style overrides.

Suggested change

# Optional style overrides for the regression line; kept empty by default.

# Optional style overrides for the regression line, initialized as an empty dictionary.

Copilot · 2025-09-18T15:45:17Z

src/emcpy/plots/map_tools.py

        """
        Sets extent, longitude xticks, and latitude yticks
-        for arctic domain.
+        for antarctic domain.


The comment mentions 'for arctic domain' in the docstring but this function is _south() which should be for 'antarctic domain'.

Copilot · 2025-09-18T15:45:17Z

src/emcpy/plots/map_plots.py

+            raise ValueError(
+                "MapScatter: latitude values exceed 90°, which suggests you passed "
+                "longitude first. Constructor order is (latitude, longitude, data)."


This validation logic for checking swapped lat/lon parameters is repeated in multiple classes (MapScatter, MapGridded, MapContour, MapFilledContour). Consider extracting this into a shared validation utility function to reduce code duplication.

Copilot · 2025-09-18T15:45:17Z

src/emcpy/plots/create_plots.py

+    ) -> None:
+


The function creates defensive copies of stats_dict and kwargs but doesn't document why this is necessary. Consider adding a docstring explaining the parameter copying behavior to avoid confusion about mutable default arguments.

Suggested change

) -> None:

) -> None:

"""

Add a dictionary of statistics to the plot, with location and formatting options.

Defensive copies of `stats_dict` and `kwargs` are made to avoid accidental mutation

of the input arguments after this method is called. This ensures that changes to the

original dictionaries outside this method do not affect the stored statistics or their

formatting in the plot.

Parameters

----------

stats_dict : Optional[Mapping[str, Any]]

Dictionary of statistics to display. A copy is made internally.

xloc : float, default 0.5

X location for the statistics text.

yloc : float, default -0.1

Y location for the statistics text.

ha : str, default "center"

Horizontal alignment.

**kwargs : Any

Additional keyword arguments for formatting. A copy is made internally.

"""

Copilot · 2025-09-18T15:45:18Z

src/emcpy/plots/create_plots.py

+            import numpy as _np  # local import to avoid any surprises at import time
+            return isinstance(v, (bool, _np.bool_)) and bool(v)


The local import of numpy as _np inside a nested function is unusual. Consider importing numpy at the module level or documenting why the local import is necessary to avoid 'surprises at import time'.

Suggested change

import numpy as _np # local import to avoid any surprises at import time

return isinstance(v, (bool, _np.bool_)) and bool(v)

return isinstance(v, (bool, np.bool_)) and bool(v)

Copilot · 2025-09-18T15:45:18Z

galleries/plot_types/statistical/violin.py

+groups = [rng.normal(0.0, 1.0, 400),
+          rng.normal(0.5, 0.8, 400),
+          rng.normal(-0.3, 1.2, 400)]


[nitpick] The gallery example uses hard-coded array sizes (400). Consider using a variable like n_samples = 400 to make the code more maintainable and self-documenting.

Suggested change

groups = [rng.normal(0.0, 1.0, 400),

rng.normal(0.5, 0.8, 400),

rng.normal(-0.3, 1.2, 400)]

n_samples = 400

groups = [rng.normal(0.0, 1.0, n_samples),

rng.normal(0.5, 0.8, n_samples),

rng.normal(-0.3, 1.2, n_samples)]

CoryMartin-NOAA

Herculean effort, @kevindougherty-noaa ! Nice work!

kevindougherty-noaa and others added 15 commits September 3, 2025 10:43

Introduce AxState + LayerAdapter (#148)

c57ac28

* introduce axstate * fix norms * pycodestyle * test CI change * test again * test * copilots reviews * pycodestyle

Centralized tick/formatter policy (datetime-safe; replaces autos with…

1b402ed

… FixedLocator) (#149) * fix tick formatting issue * pycodestyle * copilor reviews

Make subplot position detection Matplotlib-version-proof (GridSpec he…

39d1f91

…lpers) and fix single-colorbar/sharex/sharey logic (#151) * update grid position calling * copilot reviews

Standardize Box-and-Whisker API (tick_labels only), harden kwargs, an…

7bde6ac

…d stabilize scatter regression (#152) * address deprecated labels in box and whisker and other fail in Scatter * pycodestyle * copilot reviews

Expand coverage for adapters, colorbars, ticks, map transforms, and d…

ef96374

…eprecations (#155) * add more tests to stress test new additions * pynorms * cleanup vmin/vmax norm error * missed import * copilot reviews

Make Box-and-Whisker compatible with Matplotlib < 3.8 (orientation → …

6461079

…vert) (#157) * add changes to boxplot * copilot reviews

Add new plot layers, helpers, renderers, adapters, tests, and optiona…

3572e42

…l `twinx` support (#158) * Add more features * Add tests * add twinx * pycodestyle * update tests * copilot reviews * copilot reviews

quick fix to docs

6977456

attempt 2 at fixing docs

9b053a8

clean up webpage failures

faa24fe

kevindougherty-noaa changed the title ~~Feature/emc py v2~~ EMCPy V2 Sep 18, 2025

kevindougherty-noaa added 7 commits September 18, 2025 14:41

cleanup package files and update README

c6fa989

fix error

3b10dea

fixing CI again

193cea4

update doc CI yaml

e991b96

update package to fix build

f993ab6

trying again

0a09de2

add another library

cbb1141

kevindougherty-noaa requested review from ADCollard, CoryMartin-NOAA and EdwardSafford-NOAA September 18, 2025 15:40

CoryMartin-NOAA requested a review from Copilot September 18, 2025 15:43

Copilot AI reviewed Sep 18, 2025

View reviewed changes

copilot reviews

4d5118d

kevindougherty-noaa marked this pull request as ready for review September 18, 2025 16:44

kevindougherty-noaa added 9 commits September 18, 2025 16:46

missed typo

973fa6e

fix test which was failing due to incorrect order

05e8a54

fix failing test due to matplotlib version

472acac

one more catch

ddc2530

update CI tests.yaml

17f8edc

fix CI

051cfeb

try CI again

99dc9dc

test fix

3c3bdf8

test fix take two

0ffe473

CoryMartin-NOAA approved these changes Sep 23, 2025

View reviewed changes

CoryMartin-NOAA merged commit 7794574 into develop Sep 23, 2025
5 checks passed

kevindougherty-noaa deleted the feature/EMCPy_v2 branch September 24, 2025 13:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

EMCPy V2#159

EMCPy V2#159
CoryMartin-NOAA merged 32 commits intodevelopfrom
feature/EMCPy_v2

kevindougherty-noaa commented Sep 15, 2025 •

edited

Loading

Uh oh!

kevindougherty-noaa commented Sep 18, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Sep 18, 2025

Uh oh!

Copilot AI Sep 18, 2025

Uh oh!

Copilot AI Sep 18, 2025

Uh oh!

Copilot AI Sep 18, 2025

Uh oh!

Copilot AI Sep 18, 2025

Uh oh!

Copilot AI Sep 18, 2025

Uh oh!

CoryMartin-NOAA left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	# Optional style overrides for the regression line; kept empty by default.
	# Optional style overrides for the regression line, initialized as an empty dictionary.

-    ) -> None:
+    ) -> None:
+        """
+        Add a dictionary of statistics to the plot, with location and formatting options.
+        Defensive copies of `stats_dict` and `kwargs` are made to avoid accidental mutation
+        of the input arguments after this method is called. This ensures that changes to the
+        original dictionaries outside this method do not affect the stored statistics or their
+        formatting in the plot.
+        Parameters
+        ----------
+        stats_dict : Optional[Mapping[str, Any]]
+            Dictionary of statistics to display. A copy is made internally.
+        xloc : float, default 0.5
+            X location for the statistics text.
+        yloc : float, default -0.1
+            Y location for the statistics text.
+        ha : str, default "center"
+            Horizontal alignment.
+        **kwargs : Any
+            Additional keyword arguments for formatting. A copy is made internally.
+        """

		import numpy as _np # local import to avoid any surprises at import time
		return isinstance(v, (bool, _np.bool_)) and bool(v)

	import numpy as _np # local import to avoid any surprises at import time
	return isinstance(v, (bool, _np.bool_)) and bool(v)
	return isinstance(v, (bool, np.bool_)) and bool(v)

-groups = [rng.normal(0.0, 1.0, 400),
-          rng.normal(0.5, 0.8, 400),
-          rng.normal(-0.3, 1.2, 400)]
+n_samples = 400
+groups = [rng.normal(0.0, 1.0, n_samples),
+          rng.normal(0.5, 0.8, n_samples),
+          rng.normal(-0.3, 1.2, n_samples)]

Conversation

kevindougherty-noaa commented Sep 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Highlights

Breaking Changes / Migration Notes

Impact

Uh oh!

kevindougherty-noaa commented Sep 18, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

CoryMartin-NOAA left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kevindougherty-noaa commented Sep 15, 2025 •

edited

Loading