Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
8453961
feat: Preserve attributes by default in all operations
max-sixty Sep 9, 2025
fa49f34
Merge branch 'main' into keep-attrs
max-sixty Sep 10, 2025
9c224ec
Merge main and resolve conflicts in Dataset.map
max-sixty Sep 10, 2025
ab77f0c
Fix Dataset.map to properly handle coordinate attrs when keep_attrs=F…
max-sixty Sep 10, 2025
2988fe0
Optimize Dataset.map coordinate attribute handling
max-sixty Sep 10, 2025
f7f3c5b
Simplify Dataset.map attribute handling code
max-sixty Sep 10, 2025
08a4d43
Remove temporal 'now' references from comments
max-sixty Sep 12, 2025
f2e5f56
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 12, 2025
87266a2
Merge branch 'main' into keep-attrs
max-sixty Sep 12, 2025
8858170
Address remaining review comments from Stefan
max-sixty Sep 12, 2025
489f16a
Use drop_conflicts for binary operations attribute handling
max-sixty Sep 13, 2025
7f99d2b
Fix binary ops attrs: only merge when both operands have attrs
max-sixty Sep 13, 2025
f8ba047
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 13, 2025
213a563
Fix binary ops attrs handling when operands have no attrs
max-sixty Sep 13, 2025
602b169
Simplify binary ops attrs handling
max-sixty Sep 14, 2025
4f003fb
Clarify comments about attrs handling differences
max-sixty Sep 14, 2025
d8fe77d
Implement true drop_conflicts behavior for binary operations
max-sixty Sep 14, 2025
b93090f
Remove unnecessary conversion of {} to None
max-sixty Sep 14, 2025
97bdb29
Merge branch 'main' into keep-attrs
max-sixty Sep 16, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 86 additions & 0 deletions doc/whats-new.rst
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,92 @@ New Features
Breaking changes
~~~~~~~~~~~~~~~~

- **All xarray operations now preserve attributes by default** (:issue:`3891`, :issue:`2582`).
Previously, operations would drop attributes unless explicitly told to preserve them via ``keep_attrs=True``.
Additionally, when attributes are preserved in binary operations, they now combine attributes from both
operands using ``drop_conflicts`` (keeping matching attributes, dropping conflicts), instead of keeping
only the left operand's attributes.

**What changed:**

.. code-block:: python

# Before (xarray <2025.09.1):
data = xr.DataArray([1, 2, 3], attrs={"units": "meters", "long_name": "height"})
result = data.mean()
result.attrs # {} - Attributes lost!

# After (xarray ≥2025.09.1):
data = xr.DataArray([1, 2, 3], attrs={"units": "meters", "long_name": "height"})
result = data.mean()
result.attrs # {"units": "meters", "long_name": "height"} - Attributes preserved!

**Affected operations include:**

*Computational operations:*

- Reductions: ``mean()``, ``sum()``, ``std()``, ``var()``, ``min()``, ``max()``, ``median()``, ``quantile()``, etc.
- Rolling windows: ``rolling().mean()``, ``rolling().sum()``, etc.
- Groupby: ``groupby().mean()``, ``groupby().sum()``, etc.
- Resampling: ``resample().mean()``, etc.
- Weighted: ``weighted().mean()``, ``weighted().sum()``, etc.
- ``apply_ufunc()`` and NumPy universal functions

*Binary operations:*

- Arithmetic: ``+``, ``-``, ``*``, ``/``, ``**``, ``//``, ``%`` (combines attributes using ``drop_conflicts``)
- Comparisons: ``<``, ``>``, ``==``, ``!=``, ``<=``, ``>=`` (combines attributes using ``drop_conflicts``)
- With scalars: ``data * 2``, ``10 - data`` (preserves data's attributes)

*Data manipulation:*

- Missing data: ``fillna()``, ``dropna()``, ``interpolate_na()``, ``ffill()``, ``bfill()``
- Indexing/selection: ``isel()``, ``sel()``, ``where()``, ``clip()``
- Alignment: ``interp()``, ``reindex()``, ``align()``
- Transformations: ``map()``, ``pipe()``, ``assign()``, ``assign_coords()``
- Shape operations: ``expand_dims()``, ``squeeze()``, ``transpose()``, ``stack()``, ``unstack()``

**Binary operations - combines attributes with ``drop_conflicts``:**

.. code-block:: python

a = xr.DataArray([1, 2], attrs={"units": "m", "source": "sensor_a"})
b = xr.DataArray([3, 4], attrs={"units": "m", "source": "sensor_b"})
(a + b).attrs # {"units": "m"} - Matching values kept, conflicts dropped
(b + a).attrs # {"units": "m"} - Order doesn't matter for drop_conflicts

**How to restore previous behavior:**

1. **Globally for your entire script:**

.. code-block:: python

import xarray as xr

xr.set_options(keep_attrs=False) # Affects all subsequent operations

2. **For specific operations:**

.. code-block:: python

result = data.mean(dim="time", keep_attrs=False)

3. **For code blocks:**

.. code-block:: python

with xr.set_options(keep_attrs=False):
# All operations in this block drop attrs
result = data1 + data2

4. **Remove attributes after operations:**

.. code-block:: python

result = data.mean().drop_attrs()

By `Maximilian Roos <https://github.com/max-sixty>`_.

- :py:meth:`Dataset.update` now returns ``None``, instead of the updated dataset. This
completes the deprecation cycle started in version 0.17. The method still updates the
dataset in-place. (:issue:`10167`)
Expand Down
2 changes: 1 addition & 1 deletion xarray/computation/apply_ufunc.py
Original file line number Diff line number Diff line change
Expand Up @@ -1214,7 +1214,7 @@ def apply_ufunc(
func = functools.partial(func, **kwargs)

if keep_attrs is None:
keep_attrs = _get_keep_attrs(default=False)
keep_attrs = _get_keep_attrs(default=True)

if isinstance(keep_attrs, bool):
keep_attrs = "override" if keep_attrs else "drop"
Expand Down
4 changes: 2 additions & 2 deletions xarray/computation/computation.py
Original file line number Diff line number Diff line change
Expand Up @@ -701,7 +701,7 @@ def where(cond, x, y, keep_attrs=None):
* lon (lon) int64 24B 10 11 12

>>> xr.where(y.lat < 1, y, -1)
<xarray.DataArray (lat: 3, lon: 3)> Size: 72B
<xarray.DataArray 'lat' (lat: 3, lon: 3)> Size: 72B
array([[ 0. , 0.1, 0.2],
[-1. , -1. , -1. ],
[-1. , -1. , -1. ]])
Expand All @@ -726,7 +726,7 @@ def where(cond, x, y, keep_attrs=None):
from xarray.core.dataset import Dataset

if keep_attrs is None:
keep_attrs = _get_keep_attrs(default=False)
keep_attrs = _get_keep_attrs(default=True)

# alignment for three arguments is complicated, so don't support it yet
from xarray.computation.apply_ufunc import apply_ufunc
Expand Down
2 changes: 0 additions & 2 deletions xarray/computation/weighted.py
Original file line number Diff line number Diff line change
Expand Up @@ -448,7 +448,6 @@ def _weighted_quantile_1d(

result = result.transpose("quantile", ...)
result = result.assign_coords(quantile=q).squeeze()

return result

def _implementation(self, func, dim, **kwargs):
Expand Down Expand Up @@ -551,7 +550,6 @@ def _implementation(self, func, dim, **kwargs) -> DataArray:
class DatasetWeighted(Weighted["Dataset"]):
def _implementation(self, func, dim, **kwargs) -> Dataset:
self._check_dim(dim)

return self.obj.map(func, dim=dim, **kwargs)


Expand Down
4 changes: 2 additions & 2 deletions xarray/core/common.py
Original file line number Diff line number Diff line change
Expand Up @@ -1314,7 +1314,7 @@ def isnull(self, keep_attrs: bool | None = None) -> Self:
from xarray.computation.apply_ufunc import apply_ufunc

if keep_attrs is None:
keep_attrs = _get_keep_attrs(default=False)
keep_attrs = _get_keep_attrs(default=True)

return apply_ufunc(
duck_array_ops.isnull,
Expand Down Expand Up @@ -1357,7 +1357,7 @@ def notnull(self, keep_attrs: bool | None = None) -> Self:
from xarray.computation.apply_ufunc import apply_ufunc

if keep_attrs is None:
keep_attrs = _get_keep_attrs(default=False)
keep_attrs = _get_keep_attrs(default=True)

return apply_ufunc(
duck_array_ops.notnull,
Expand Down
4 changes: 2 additions & 2 deletions xarray/core/dataarray.py
Original file line number Diff line number Diff line change
Expand Up @@ -3889,8 +3889,8 @@ def reduce(
supplied, then the reduction is calculated over the flattened array
(by calling `f(x)` without an axis argument).
keep_attrs : bool or None, optional
If True, the variable's attributes (`attrs`) will be copied from
the original object to the new one. If False (default), the new
If True (default), the variable's attributes (`attrs`) will be copied from
the original object to the new one. If False, the new
object will be returned without attributes.
keepdims : bool, default: False
If True, the dimensions which are reduced are left in the result
Expand Down
32 changes: 20 additions & 12 deletions xarray/core/dataset.py
Original file line number Diff line number Diff line change
Expand Up @@ -6774,8 +6774,8 @@ def reduce(
Dimension(s) over which to apply `func`. By default `func` is
applied over all dimensions.
keep_attrs : bool or None, optional
If True, the dataset's attributes (`attrs`) will be copied from
the original object to the new one. If False (default), the new
If True (default), the dataset's attributes (`attrs`) will be copied from
the original object to the new one. If False, the new
object will be returned without attributes.
keepdims : bool, default: False
If True, the dimensions which are reduced are left in the result
Expand Down Expand Up @@ -6833,7 +6833,7 @@ def reduce(
dims = parse_dims_as_set(dim, set(self._dims.keys()))

if keep_attrs is None:
keep_attrs = _get_keep_attrs(default=False)
keep_attrs = _get_keep_attrs(default=True)

variables: dict[Hashable, Variable] = {}
for name, var in self._variables.items():
Expand Down Expand Up @@ -6924,7 +6924,7 @@ def map(
bar (x) float64 16B 1.0 2.0
"""
if keep_attrs is None:
keep_attrs = _get_keep_attrs(default=False)
keep_attrs = _get_keep_attrs(default=True)
variables = {
k: maybe_wrap_array(v, func(v, *args, **kwargs))
for k, v in self.data_vars.items()
Expand All @@ -6937,11 +6937,14 @@ def map(
if keep_attrs:
for k, v in variables.items():
v._copy_attrs_from(self.data_vars[k])

for k, v in coords.items():
if k not in self.coords:
continue
v._copy_attrs_from(self.coords[k])
if k in self.coords:
v._copy_attrs_from(self.coords[k])
else:
for v in variables.values():
v.attrs = {}
for v in coords.values():
v.attrs = {}

attrs = self.attrs if keep_attrs else None
return type(self)(variables, coords=coords, attrs=attrs)
Expand Down Expand Up @@ -7669,9 +7672,14 @@ def _binary_op(self, other, f, reflexive=False, join=None) -> Dataset:
self, other = align(self, other, join=align_type, copy=False)
g = f if not reflexive else lambda x, y: f(y, x)
ds = self._calculate_binary_op(g, other, join=align_type)
keep_attrs = _get_keep_attrs(default=False)
keep_attrs = _get_keep_attrs(default=True)
if keep_attrs:
ds.attrs = self.attrs
# Combine attributes from both operands, dropping conflicts
from xarray.structure.merge import merge_attrs

self_attrs = self.attrs
other_attrs = getattr(other, "attrs", {})
ds.attrs = merge_attrs([self_attrs, other_attrs], "drop_conflicts")
return ds

def _inplace_binary_op(self, other, f) -> Self:
Expand Down Expand Up @@ -8265,7 +8273,7 @@ def quantile(
coord_names = {k for k in self.coords if k in variables}
indexes = {k: v for k, v in self._indexes.items() if k in variables}
if keep_attrs is None:
keep_attrs = _get_keep_attrs(default=False)
keep_attrs = _get_keep_attrs(default=True)
attrs = self.attrs if keep_attrs else None
new = self._replace_with_new_dims(
variables, coord_names=coord_names, attrs=attrs, indexes=indexes
Expand Down Expand Up @@ -8327,7 +8335,7 @@ def rank(

coord_names = set(self.coords)
if keep_attrs is None:
keep_attrs = _get_keep_attrs(default=False)
keep_attrs = _get_keep_attrs(default=True)
attrs = self.attrs if keep_attrs else None
return self._replace(variables, coord_names, attrs=attrs)

Expand Down
2 changes: 1 addition & 1 deletion xarray/core/datatree.py
Original file line number Diff line number Diff line change
Expand Up @@ -428,7 +428,7 @@ def map( # type: ignore[override]
# Copied from xarray.Dataset so as not to call type(self), which causes problems (see https://github.com/xarray-contrib/datatree/issues/188).
# TODO Refactor xarray upstream to avoid needing to overwrite this.
if keep_attrs is None:
keep_attrs = _get_keep_attrs(default=False)
keep_attrs = _get_keep_attrs(default=True)
variables = {
k: maybe_wrap_array(v, func(v, *args, **kwargs))
for k, v in self.data_vars.items()
Expand Down
27 changes: 18 additions & 9 deletions xarray/core/variable.py
Original file line number Diff line number Diff line change
Expand Up @@ -1741,8 +1741,8 @@ def reduce( # type: ignore[override]
the reduction is calculated over the flattened array (by calling
`func(x)` without an axis argument).
keep_attrs : bool, optional
If True, the variable's attributes (`attrs`) will be copied from
the original object to the new one. If False (default), the new
If True (default), the variable's attributes (`attrs`) will be copied from
the original object to the new one. If False, the new
object will be returned without attributes.
keepdims : bool, default: False
If True, the dimensions which are reduced are left in the result
Expand All @@ -1757,7 +1757,7 @@ def reduce( # type: ignore[override]
removed.
"""
keep_attrs_ = (
_get_keep_attrs(default=False) if keep_attrs is None else keep_attrs
_get_keep_attrs(default=True) if keep_attrs is None else keep_attrs
)

# Note that the call order for Variable.mean is
Expand Down Expand Up @@ -2009,7 +2009,7 @@ def quantile(
_quantile_func = duck_array_ops.quantile

if keep_attrs is None:
keep_attrs = _get_keep_attrs(default=False)
keep_attrs = _get_keep_attrs(default=True)

scalar = utils.is_scalar(q)
q = np.atleast_1d(np.asarray(q, dtype=np.float64))
Expand Down Expand Up @@ -2350,7 +2350,7 @@ def isnull(self, keep_attrs: bool | None = None):
from xarray.computation.apply_ufunc import apply_ufunc

if keep_attrs is None:
keep_attrs = _get_keep_attrs(default=False)
keep_attrs = _get_keep_attrs(default=True)

return apply_ufunc(
duck_array_ops.isnull,
Expand Down Expand Up @@ -2384,7 +2384,7 @@ def notnull(self, keep_attrs: bool | None = None):
from xarray.computation.apply_ufunc import apply_ufunc

if keep_attrs is None:
keep_attrs = _get_keep_attrs(default=False)
keep_attrs = _get_keep_attrs(default=True)

return apply_ufunc(
duck_array_ops.notnull,
Expand Down Expand Up @@ -2435,8 +2435,17 @@ def _binary_op(self, other, f, reflexive=False):
other_data, self_data, dims = _broadcast_compat_data(other, self)
else:
self_data, other_data, dims = _broadcast_compat_data(self, other)
keep_attrs = _get_keep_attrs(default=False)
attrs = self._attrs if keep_attrs else None
keep_attrs = _get_keep_attrs(default=True)
if keep_attrs:
# Combine attributes from both operands, dropping conflicts
from xarray.structure.merge import merge_attrs

# Access attrs property to normalize None to {} due to property side effect
self_attrs = self.attrs
other_attrs = getattr(other, "attrs", {})
attrs = merge_attrs([self_attrs, other_attrs], "drop_conflicts")
else:
attrs = None
with np.errstate(all="ignore"):
new_data = (
f(self_data, other_data) if not reflexive else f(other_data, self_data)
Expand Down Expand Up @@ -2526,7 +2535,7 @@ def _unravel_argminmax(
}

if keep_attrs is None:
keep_attrs = _get_keep_attrs(default=False)
keep_attrs = _get_keep_attrs(default=True)
if keep_attrs:
for v in result.values():
v.attrs = self.attrs
Expand Down
4 changes: 2 additions & 2 deletions xarray/tests/test_coarsen.py
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ def test_coarsen_keep_attrs(funcname, argument) -> None:
attrs=global_attrs,
)

# attrs are now kept per default
# attrs are kept by default
func = getattr(ds.coarsen(dim={"coord": 5}), funcname)
result = func(*argument)
assert result.attrs == global_attrs
Expand Down Expand Up @@ -199,7 +199,7 @@ def test_coarsen_da_keep_attrs(funcname, argument) -> None:
name="name",
)

# attrs are now kept per default
# attrs are kept by default
func = getattr(da.coarsen(dim={"coord": 5}), funcname)
result = func(*argument)
assert result.attrs == attrs_da
Expand Down
Loading
Loading