Merge branch 'main' into docs/issue-4159-array-bytes-codec

Author: d-v-b

.github/workflows/needs_release_notes.yml

@@ -21,7 +21,7 @@ jobs:
       pull-requests: write # Required to add labels to PRs
     runs-on: ubuntu-latest
     steps:
-    - uses: actions/labeler@b8dd2d9be0f68b860e7dae5dae7d772984eacd6d  # v6.2.0
+    - uses: actions/labeler@bf12e9b00b37c5c0ca2b87b79b2daf7891dbda13  # v7.0.0
       with:
         repo-token: ${{ secrets.GITHUB_TOKEN }}
         sync-labels: true

.github/workflows/releases.yml

@@ -81,7 +81,7 @@ jobs:
           name: releases
           path: dist
       - name: Generate artifact attestation
-        uses: actions/attest@a1948c3f048ba23858d222213b7c278aabede763 # v4.1.1
+        uses: actions/attest@f7c74d28b9d84cb8768d0b8ca14a4bac6ef463e6 # v4.2.0
         with:
           subject-path: dist/*
       - name: Publish package to PyPI

.github/workflows/zarr-metadata-release.yml

@@ -82,7 +82,7 @@ jobs:
           path: dist
 
       - name: Generate artifact attestation
-        uses: actions/attest@a1948c3f048ba23858d222213b7c278aabede763 # v4.1.1
+        uses: actions/attest@f7c74d28b9d84cb8768d0b8ca14a4bac6ef463e6 # v4.2.0
         with:
           subject-path: dist/*
 
@@ -107,7 +107,7 @@ jobs:
           path: dist
 
       - name: Generate artifact attestation
-        uses: actions/attest@a1948c3f048ba23858d222213b7c278aabede763 # v4.1.1
+        uses: actions/attest@f7c74d28b9d84cb8768d0b8ca14a4bac6ef463e6 # v4.2.0
         with:
           subject-path: dist/*
 

.github/workflows/zarr-metadata.yml

@@ -82,7 +82,10 @@ jobs:
       - name: Sync test dependency group
         run: uv sync --group test --python 3.11
       - name: Run pyright
-        run: uv run --group test --with pyright pyright src
+        # Pinned to the last version that types PEP 661 sentinels in class
+        # attributes correctly; 1.1.405+ regressed (microsoft/pyright#11115).
+        # Unpin when the fix lands.
+        run: uv run --group test --with 'pyright==1.1.404' pyright src
 
   zarr-metadata-complete:
     name: zarr-metadata complete

.github/workflows/zizmor.yml

@@ -32,4 +32,4 @@ jobs:
           persist-credentials: false
 
       - name: Run zizmor
-        uses: zizmorcore/zizmor-action@192e21d79ab29983730a13d1382995c2307fbcaa # v0.5.7
+        uses: zizmorcore/zizmor-action@6599ee8b7a49aef6a770f63d261d214911a7ce02 # v0.6.0

changes/4128.feature.md

@@ -0,0 +1 @@
+Added `Group.get_array`, `Group.get_group`, `AsyncGroup.get_array`, and `AsyncGroup.get_group`: type-safe accessors that return the child array or group at a given path, raising `ArrayNotFoundError` / `GroupNotFoundError` if no node exists there, and `ContainsGroupError` / `ContainsArrayError` if the node is not of the requested kind. Unlike `Group.__getitem__`, which returns `Array | Group`, these methods have precise return types. Nested paths like `"subgroup/subarray"` are supported.

changes/4187.feature.md

@@ -0,0 +1,4 @@
+`ZipStore` now accepts an open binary file-like object in place of a path, enabling
+zip archives on remote storage (e.g. a file opened with `fsspec` or an
+`obstore.ReadableFile`). Operations that require a filesystem location
+(`clear`, `move`) raise `NotImplementedError` for file-object-backed stores.

docs/user-guide/groups.md

@@ -51,6 +51,29 @@ print(root['foo/bar'])
 print(root['foo/bar/baz'])
 ```
 
+Accessing a member with `[]` returns either an [`zarr.Array`][] or a [`zarr.Group`][], depending on
+what is stored at the given path. When you expect a node of a particular kind, use
+[`zarr.Group.get_array`][] or [`zarr.Group.get_group`][] instead. These methods accept the same
+paths as `[]`, but they have precise return types and raise an error if no node exists at the
+given path, or if the node is not of the expected kind:
+
+```python exec="true" session="groups" source="above" result="ansi"
+print(root.get_group('foo'))
+```
+
+```python exec="true" session="groups" source="above" result="ansi"
+print(root.get_array('foo/bar/baz'))
+```
+
+```python exec="true" session="groups" source="above" result="ansi"
+from zarr.errors import ContainsGroupError
+
+try:
+    root.get_array('foo')
+except ContainsGroupError as e:
+    print(e)
+```
+
 The [`zarr.Group.tree`][] method can be used to print a tree
 representation of the hierarchy, e.g.:
 

docs/user-guide/storage.md

@@ -124,6 +124,19 @@ array = zarr.create_array(store=store, shape=(2,), dtype='float64')
 print(array)
 ```
 
+In place of a path, `ZipStore` also accepts an open binary file object (for
+example a file opened with `fsspec`, or an `obstore` reader), enabling zip
+archives on remote storage. The file must stay open for as long as the store
+is in use:
+
+```python exec="true" session="storage" source="above" result="ansi"
+store.close()
+f = open('data.zip', mode='rb')  # must stay open while the store is used
+array = zarr.open_array(store=zarr.storage.ZipStore(f), mode='r')
+print(array[:])
+f.close()
+```
+
 ### Remote Store
 
 The [`zarr.storage.FsspecStore`][] stores the contents of a Zarr hierarchy following the same

packages/zarr-metadata/README.md

@@ -1,51 +1,86 @@
 # zarr-metadata
 
-Python type definitions for Zarr v2 and v3 metadata.
+Python types, models, and validators for Zarr v2 and v3 metadata.
 
 ## What this is
 
-A typed-data package: `TypedDict` definitions and `Literal` aliases for the
-JSON shapes specified by the [Zarr v2](https://zarr-specs.readthedocs.io/en/latest/v2/v2.0.html)
-and [Zarr v3](https://zarr-specs.readthedocs.io/en/latest/v3/core/index.html)
-specifications, plus types for [`zarr-extensions`](https://github.com/zarr-developers/zarr-extensions/)
-and a few widely-used-but-unspecified entities (e.g. consolidated metadata).
+Two layers and an optional integration:
+
+- **Typed JSON shapes**: `TypedDict` definitions and `Literal` aliases for the
+  JSON documents specified by the [Zarr v2](https://zarr-specs.readthedocs.io/en/latest/v2/v2.0.html)
+  and [Zarr v3](https://zarr-specs.readthedocs.io/en/latest/v3/core/index.html)
+  specifications, plus types for [`zarr-extensions`](https://github.com/zarr-developers/zarr-extensions/)
+  and a few widely-used-but-unspecified entities (e.g. consolidated metadata).
+- **Document models** (`zarr_metadata.model`): canonical frozen-dataclass
+  models of whole metadata documents, with structural validators, loc-aware
+  parsers, and store-key (de)serialization. A document produced by `to_json`
+  shares no mutable state with the model that produced it.
+- **Optional Pydantic integration** (`zarr_metadata.pydantic`, requires
+  Pydantic 2.13 or newer): each model as a Pydantic field type that validates
+  raw documents through the same strict parser.
 
 ## What this is for
 
-These types describe the JSON shape of Zarr metadata. They are
-intended for libraries that **read, write, validate, or transform**
-Zarr metadata. Pair them with a runtime validator like
-[pydantic](https://docs.pydantic.dev/) to check JSON loaded from disk:
+The public `TypedDict` definitions describe the static JSON shape of Zarr
+metadata. For strict, loc-aware validation of JSON loaded from disk, use the
+model parser:
 
 ```python
 import json
-from pydantic import TypeAdapter
-from zarr_metadata.v3.array import ArrayMetadataV3
+from zarr_metadata.model import ZarrV3ArrayMetadata
 
 with open("zarr.json", "rb") as f:
     raw = json.load(f)
 
-metadata = TypeAdapter(ArrayMetadataV3).validate_python(raw)
+metadata = ZarrV3ArrayMetadata.from_json(raw)
+```
+
+The optional Pydantic integration delegates raw input to the same strict
+parser and returns the same normalized model class:
+
+```python
+from pydantic import TypeAdapter
+import zarr_metadata.pydantic as zmp
+
+metadata = TypeAdapter(zmp.ZarrV3ArrayMetadata).validate_python(raw)
+encoded = metadata.to_key_value()["zarr.json"]
 ```
 
-## What this is *not*
+A bare `TypeAdapter` over a public document `TypedDict` is a coercive shape
+adapter, not a Zarr conformance validator; it may coerce values or discard
+members that the strict model parser rejects.
+
+## Validation boundary
 
-- Not a parser or builder. There are no `make_array_metadata(...)` factories —
-  that surface belongs to consumer libraries.
-- Not a runtime validator on its own. Pair with `pydantic`, `msgspec`, or
-  similar to enforce shapes at decode time.
+The model validators enforce the declared document structure and a small set
+of context-free consistency rules, including fixed format literals, finite
+JSON numbers, non-negative dimensions, non-empty v3 codec pipelines, and one
+`dimension_names` entry per array dimension. They do not interpret extension
+names or configurations, resolve codec pipelines, or decide whether a data
+type, chunk grid, codec, or storage transformer is supported. Those decisions
+belong to consumer implementations.
 
-Even with a runtime validator, these types only describe **structural**
-shape — they will not flag *semantically* invalid metadata, like a 3D v3
-array whose `dimension_names` has 4 entries instead of 3. That's a job
-for downstream validator routines.
+The Pydantic integration's generated JSON Schemas express independently
+checkable document structure and field constraints, but they are not a
+replacement for runtime model validation. Standard JSON Schema treats a
+mathematically integral number such as `1.0` as an integer, while the runtime
+boundary requires Python `int` values, and it cannot express arbitrary
+same-length relations such as `dimension_names` versus `shape` or v2 `chunks`
+versus `shape`. Consumers should run the model parser after schema validation.
 
 ## Scope
 
 At minimum, this library supports what Zarr-Python needs: the complete
 Zarr v2 and v3 specs, consolidated metadata, and a subset of the metadata
 defined in `zarr-extensions`. We are generally open to contributions that
-add types for Zarr metadata with a published spec.
+add types, models, or structural validation for Zarr metadata with a
+published spec.
+
+Runtime array behavior is out of scope: nothing here encodes or decodes
+chunks, resolves codec or data type names to implementations, or performs
+store I/O. The models begin and end at the metadata documents themselves —
+`from_key_value` / `to_key_value` map documents to store keys and bytes,
+and everything past that belongs to consumer libraries.
 
 ## Releasing