Skip to content

Commit 0fb6d52

Browse files
committed
Fix
1 parent ff96bf0 commit 0fb6d52

10 files changed

Lines changed: 434 additions & 7 deletions

File tree

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ Cellucid is a browser-first viewer for exploring large single-cell datasets in r
1616
- **GPU-accelerated WebGL rendering** with level-of-detail culling; a built-in
1717
synthetic benchmark measures what your own GPU sustains
1818
- **AnnData-first workflow** for the Scanpy ecosystem (`.h5ad` and Zarr supported)
19-
- **Shareable exports** you can host locally, on GitHub, or behind a server
19+
- **Shareable exports** you can host locally, on GitHub, on any web host, as a single-file pack (`pack()`), or behind a server
2020
- **Genes + metadata overlays** optimized for interactive querying
2121
- **Connectivity + dynamics**: KNN edges and animated vector fields (RNA velocity / drift)
2222
- **Collaboration**: community annotation voting with optional GitHub sync

‎docs/user_guide/python_package/c_data_preparation_api/09_output_format_specification_exports_directory.md‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1130,6 +1130,47 @@ Web app docs:
11301130

11311131
---
11321132

1133+
## Single-file pack (`pack()`, `cellucid_pack()`)
1134+
1135+
A **pack** is one ZIP file holding exactly the files of one dataset's export
1136+
directory. It exists for hosts that serve single files rather than folders (a
1137+
Zenodo record, a cloud-drive or Dropbox file link, a signed bucket URL): the web
1138+
app opens it from any web address with **Open from link** or `?data=<address>`,
1139+
reads its central directory once, and then reads each file with one HTTP range
1140+
request when a view needs it. Nothing is unpacked or downloaded whole.
1141+
1142+
```python
1143+
from cellucid import pack
1144+
1145+
pack("exports/pbmc") # writes exports/pbmc.zip
1146+
```
1147+
1148+
```r
1149+
cellucid_pack("exports/pbmc") # writes exports/pbmc.zip
1150+
```
1151+
1152+
The bytes depend only on the directory's contents, and the two writers write
1153+
the identical file (both test suites pin the digest of the same pack):
1154+
1155+
- every regular file under the directory, at its relative `/`-separated path,
1156+
in ascending code-point order of path; no directory entries, no symbolic
1157+
links;
1158+
- each entry stored uncompressed (method 0: the payloads are already gzip
1159+
members), with the UTF-8 flag (bit 11), the DOS time 1980-01-01 00:00:00,
1160+
version needed 2.0 (4.5 for an entry that carries ZIP64 fields), version made
1161+
by Unix with the same version, external attributes of a `0644` regular file,
1162+
no comment, no data descriptor, and no extra field unless a ZIP64 one is
1163+
needed;
1164+
- a ZIP64 extra field exactly where a size or an offset does not fit its 32-bit
1165+
field, and a ZIP64 end record and locator exactly when the entry count does
1166+
not fit 16 bits or the central directory's size or offset does not fit 32.
1167+
1168+
Cellucid recognises a pack by the `dataset_identity.json` at its root; any other
1169+
`.zip` is read as a zipped Zarr store. A catalog root (with `datasets.json`) is
1170+
not packed: pack each dataset folder, and share each pack's address. A pack
1171+
opened from a link is the same dataset as its folder: same identity id, so
1172+
sessions and community annotations carry over.
1173+
11331174
## Edge cases and common footguns
11341175

11351176
- **Rejected candidates**: generation publication is atomic; a failed or

‎docs/user_guide/python_package/g_api_reference_coverage/api/export.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -156,6 +156,15 @@ Notes:
156156
.. autofunction:: prepare
157157
```
158158

159+
To share one dataset from a host that serves single files (a Zenodo record, a
160+
file link, a signed bucket URL), write it as a single-file pack; the format is
161+
in {doc}`../../c_data_preparation_api/09_output_format_specification_exports_directory`
162+
("Single-file pack").
163+
164+
```{eval-rst}
165+
.. autofunction:: cellucid.pack
166+
```
167+
159168
---
160169

161170
## Edge cases (do not skip)

‎docs/user_guide/r_package/d_viewing_loading/02_host_exports_for_sharing.md‎

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -137,11 +137,27 @@ Create a public GitHub repo and commit:
137137
- `exports/datasets.json`
138138
- `exports/<dataset_folder>/...`
139139

140-
Then, in the Cellucid web app, use the GitHub loading option and point it at your repo/path.
140+
Then, in the Cellucid web app, paste `owner/repo/exports` into **Open from link**,
141+
or share `https://www.cellucid.com/?data=gh:owner/repo/exports&dataset=<id>`.
141142

142143
The authoritative step-by-step is here:
143144
- {doc}`../../web_app/b_data_loading/11_custom_dataset_repository`
144145

146+
## Option 1b — One dataset as a single file (any host)
147+
148+
A host that serves single files rather than folders (a Zenodo record, a file
149+
link, a signed bucket URL) can serve a prepared export as one **pack**:
150+
151+
```r
152+
cellucid_pack("exports/my_dataset") # writes exports/my_dataset.zip
153+
```
154+
155+
Upload the `.zip` and open its web address with **Open from link**, or share
156+
`https://www.cellucid.com/?data=<address>`. Cellucid reads it in place, one file
157+
per view, and the pack is byte-identical to the one `pack()` in the Python
158+
package writes. Every way to open data from a link:
159+
{doc}`../../web_app/b_data_loading/14_open_from_link`.
160+
145161
## Option 2 — Private sharing via server mode
146162

147163
If your dataset is private (patient IDs, unpublished experiments, etc.), do not publish to GitHub.

‎docs/user_guide/web_app/b_data_loading/14_open_from_link.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ shows its embedding after reading about 15 MB.
3434
| A GEO series (`GSE123456`), its accession page, or a supplementary file link | The series' `.h5ad` supplementary file | `geo:GSE123456` or `geo:GSE123456/<file>.h5ad` |
3535
| A GitHub repository path (`owner/repo/exports`, `owner/repo@branch/exports`), a `github.com/.../tree/...` link, or a `raw.githubusercontent.com` link | The prepared-export catalog in that folder ({doc}`11_custom_dataset_repository`) | `gh:owner/repo/exports` |
3636
| A web address ending in `/`, or its `datasets.json` | The prepared-export catalog in that folder | the address itself |
37+
| A web address of a prepared-export **pack** (one dataset in one `.zip`, written by `pack()` in Python or `cellucid_pack()` in R) | That dataset, read file by file in place | the address itself |
3738
| A web address of an `.h5ad` file or a Zarr `.zip`: a bucket, a lab server, a Hugging Face file link (`huggingface.co/datasets/<user>/<repo>/resolve/main/<file>`), or a Zenodo API file link (`zenodo.org/api/records/<id>/files/<file>/content`; Zenodo's ordinary download link cannot be read by other sites) | That file | the address itself |
3839

3940
A link that holds several datasets (a catalog) lists them under **Linked data**

‎src/cellucid/__init__.py‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66
77
Main exports:
88
- prepare: Export data to cellucid format
9+
- pack: Write one prepared dataset as a single-file pack
910
- Embedding: One embedding to export, with the columns it is read from
1011
- serve: Serve a dataset directory via HTTP
1112
- CellucidServer: Server class for more control
@@ -33,7 +34,7 @@
3334
from .anndata_server import AnnDataServer, serve_anndata
3435
from .anndata_session import apply_cellucid_session_to_anndata
3536
from .jupyter import AnnDataViewer, CellucidViewer, show, show_anndata
36-
from .prepare_data import Embedding, prepare
37+
from .prepare_data import Embedding, pack, prepare
3738
from .server import CellucidServer, serve
3839
from .session_bundle import CellucidSessionBundle
3940
from .vector_fields import add_transition_drift_to_obsm, compute_transition_drift
@@ -52,6 +53,10 @@ def __getattr__(name: str) -> Any:
5253
from .prepare_data import Embedding
5354

5455
return Embedding
56+
if name == "pack":
57+
from .prepare_data import pack
58+
59+
return pack
5560

5661
# Server for pre-exported data (lightweight, no heavy deps)
5762
if name == "serve":
@@ -131,6 +136,7 @@ def __getattr__(name: str) -> Any:
131136
__all__ = [
132137
# Core export functionality
133138
"prepare",
139+
"pack",
134140
"Embedding",
135141
# Serve pre-exported data
136142
"serve",

‎src/cellucid/prepare_data/__init__.py‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@
1818
- var_gene_name_columns: further var columns offered as alternative gene names
1919
- connectivities: sparse matrix with KNN connectivities
2020
21-
The module is a package. Its public surface is exactly the two functions and the
21+
The module is a package. Its public surface is exactly the three functions and the
2222
one class below; everything else is split by responsibility across private
2323
submodules:
2424
@@ -39,12 +39,14 @@
3939
exclusive writer lock.
4040
``_generation``
4141
The writer that builds one complete generation in a staging directory.
42-
``_export`` / ``_catalog``
43-
:func:`prepare` and :func:`generate_datasets_manifest`.
42+
``_export`` / ``_catalog`` / ``_pack``
43+
:func:`prepare`, :func:`generate_datasets_manifest`, and :func:`pack`
44+
(one dataset as a single-file pack).
4445
"""
4546

4647
from ._catalog import generate_datasets_manifest
4748
from ._embeddings import Embedding
4849
from ._export import prepare
50+
from ._pack import pack
4951

50-
__all__ = ["Embedding", "generate_datasets_manifest", "prepare"]
52+
__all__ = ["Embedding", "generate_datasets_manifest", "pack", "prepare"]

0 commit comments

Comments
 (0)