Merge pull request #83 from PyAutoLabs/feature/cti-resurrection-phase0

CTI resurrection Phase 0: resurrect against the current PyAuto stack

Author: Jammy2211

AGENTS.md

@@ -0,0 +1,112 @@
+# PyAutoCTI — Agent Instructions
+
+Canonical, agent-agnostic instructions for this repo. `CLAUDE.md` imports this
+file; any tool that does not process `@`-imports should read this directly.
+
+## What this repo is
+
+**PyAutoCTI** (package `autocti`) is a Bayesian library for calibrating and
+modelling Charge Transfer Inefficiency (CTI) in CCD imaging: charge-injection
+imaging (`ImagingCI`) and 1D datasets (`Dataset1D`), trap/CCD models clocked
+through the C++ **arctic** code (`Clocker1D`/`Clocker2D` wrapping `arcticpy`),
+FPR/EPER extraction (`autocti/extract/`), and per-dataset `Fit*`/`Analysis*`
+classes. Heritage: Euclid VIS CTI calibration; also HST ACS
+(`autocti/instruments/acs`).
+
+Dependency direction: autocti may import **autoarray** (data structures),
+**autofit** (model-fitting), and **autoconf** (config). Nothing in the PyAuto
+stack imports autocti — it is a leaf like PyAutoLens.
+
+## Resurrection status (2026-07)
+
+This repo was unmaintained for ~2 years and is being brought back into the
+ecosystem via the CTI resurrection epic
+([PyAutoCTI#82](https://github.com/PyAutoLabs/PyAutoCTI/issues/82)). Phase 0
+(importable + unit tests green on the current stack) is complete. **The
+visualization layer (`autocti/plot/`, `*/plot/*_plotters.py`,
+`*/model/plotter_interface.py`) is quarantined**: it still targets the removed
+autoarray Plotter API and is rewritten on the matplotlib function API
+(mirroring PyAutoGalaxy) in Phase 1. Until then `autocti.plot` is not
+importable, `Analysis` visualization no-ops with a logged warning, and the
+plot tests are skipped via `test_autocti/conftest.py`.
+
+## arcticpy (read before installing)
+
+`import autocti` requires **arcticpy** (pinned 2.6), which is deliberately not
+a pip dependency:
+
+- Its PyPI sdist is **source-only C++** — it needs `libgsl-dev` headers and a
+  toolchain to build.
+- Its own requirements **downgrade numpy below 2.0**, breaking a modern stack.
+
+Install it after numpy is in place:
+
+```bash
+pip install arcticpy==2.6 --no-build-isolation --no-deps
+```
+
+If GSL headers are missing and you lack root, extract them locally
+(`apt-get download libgsl-dev && dpkg -x ...`) and point `CPPFLAGS`/`LDFLAGS`
+at them.
+
+## Quick commands
+
+```bash
+pip install -e ".[dev]"                                  # install with dev/test extras
+python -m pytest test_autocti/                           # full test suite
+python -m pytest test_autocti/extract/                   # one focused directory
+```
+
+In a sandboxed / restricted environment, point numba and matplotlib at
+writable caches:
+
+```bash
+NUMBA_CACHE_DIR=/tmp/numba_cache MPLCONFIGDIR=/tmp/matplotlib python -m pytest test_autocti/
+```
+
+## Related repos
+
+- **Source siblings:** PyAutoConf, PyAutoArray, PyAutoFit (upstream).
+- **autocti_workspace** — runnable examples/tutorials (updated in epic Phase 4).
+- **autocti_workspace_test** — regression scripts + Euclid tvac/temporal
+  heritage (rebuilt in epic Phase 5).
+- **Science context:** `PyAutoMemory/wiki/cti/` (trap physics, arctic
+  algorithm, Euclid VIS / HST ACS heritage).
+
+## Public API
+
+The public surface is defined authoritatively in `autocti/__init__.py` — read
+it rather than trusting a hand-maintained table. Canonical import:
+
+```python
+import autocti as ac
+```
+
+## Key rules / footguns
+
+- Import direction: autoarray / autofit / autoconf only — never autogalaxy or
+  autolens.
+- Unit tests are numpy-only; there is no JAX in this library (arctic is C++).
+- Slicing an autoarray `Mask2D` returns a plain ndarray — rebuild a `Mask2D`
+  with the parent's `pixel_scales` before constructing an `Array2D` from it
+  (see `autocti/extract/two_d/abstract.py`).
+- Fits I/O goes through `autoconf.fitsable` (`ndarray_via_fits_from`,
+  `output_to_fits`, `hdu_list_for_output_from`) — instance `.output_to_fits`
+  methods no longer exist on autoarray structures.
+- All files use Unix line endings (LF, `\n`) — never `\r\n`.
+
+## Working on issues
+
+1. Read the issue description and any linked plan.
+2. Identify affected files and make the change.
+3. Run the full suite: `python -m pytest test_autocti/`.
+4. If you changed public API, say so explicitly — autocti_workspace may need
+   updates.
+5. Ensure all tests pass before opening a PR.
+
+## Never rewrite history
+
+Never rewrite pushed history on any repo with a remote — no `git init` over a
+tracked repo, no force-push to `main`, no fresh-start "Initial commit", no
+`filter-repo` / `filter-branch` / `rebase -i` on pushed branches. To get a
+clean tree: `git fetch origin && git reset --hard origin/main && git clean -fd`.

CLAUDE.md

@@ -0,0 +1,5 @@
+# PyAutoCTI — agent instructions
+The canonical, agent-agnostic instructions live in `AGENTS.md`. Claude Code loads them
+via the import below; if your tool does not process `@`-imports, open `AGENTS.md` in
+this directory and read it directly.
+@AGENTS.md

autocti/__init__.py

@@ -71,7 +71,10 @@
 from .clocker.two_d import Clocker2D
 from . import aggregator as agg
 from . import util
-from . import plot
+
+# `from . import plot` is quarantined: the Plotter object stack targets the
+# removed autoarray Plotter API and is rewritten on the new matplotlib function
+# API in Phase 1 of the CTI resurrection epic (PyAutoCTI#82).
 from . import mock as m  # noqa
 
 from autoconf import conf

autocti/aggregator/dataset_1d.py

@@ -3,6 +3,7 @@
 
 import autofit as af
 import autoarray as aa
+from autoconf.fitsable import ndarray_via_hdu_from
 
 from autocti.dataset_1d.dataset_1d.dataset_1d import Dataset1D
 
@@ -54,23 +55,28 @@ def _dataset_1d_list_from(
     for fit in fit_list:
         layout = fit.value(name=f"{folder}.layout")
 
-        data = aa.Array1D.from_primary_hdu(primary_hdu=fit.value(name=f"{folder}.data"))
-        noise_map = aa.Array1D.from_primary_hdu(
-            primary_hdu=fit.value(name=f"{folder}.noise_map")
-        )
-        pre_cti_data = aa.Array1D.from_primary_hdu(
-            primary_hdu=fit.value(name=f"{folder}.pre_cti_data")
+        hdu_list = fit.value(name=f"{folder}.dataset")
+
+        pixel_scales = hdu_list[0].header["PIXSCA"]
+
+        mask = aa.Mask1D(
+            mask=ndarray_via_hdu_from(hdu_list[0]).astype("bool"),
+            pixel_scales=pixel_scales,
         )
 
+        def values_from(hdu: int) -> aa.Array1D:
+            return aa.Array1D.no_mask(
+                values=ndarray_via_hdu_from(hdu_list[hdu]),
+                pixel_scales=pixel_scales,
+            )
+
         dataset = Dataset1D(
-            data=data,
-            noise_map=noise_map,
-            pre_cti_data=pre_cti_data,
+            data=values_from(hdu=1),
+            noise_map=values_from(hdu=2),
+            pre_cti_data=values_from(hdu=3),
             layout=layout,
         )
 
-        mask = aa.Mask1D.from_primary_hdu(primary_hdu=fit.value(name=f"{folder}.mask"))
-
         dataset_list.append(dataset.apply_mask(mask=mask))
 
     return dataset_list

autocti/aggregator/imaging_ci.py

@@ -1,6 +1,7 @@
 from functools import partial
 
 import autoarray as aa
+from autoconf.fitsable import ndarray_via_hdu_from
 import autofit as af
 
 from autocti.charge_injection.imaging.imaging import ImagingCI
@@ -56,33 +57,40 @@ def _imaging_ci_list_from(fit: af.Fit, use_dataset_full: bool = False):
     for fit in fit_list:
         layout = fit.value(name=f"{folder}.layout")
 
-        data = aa.Array2D.from_primary_hdu(primary_hdu=fit.value(name=f"{folder}.data"))
-        noise_map = aa.Array2D.from_primary_hdu(
-            primary_hdu=fit.value(name=f"{folder}.noise_map")
+        hdu_list = fit.value(name=f"{folder}.dataset")
+
+        pixel_scales = (
+            hdu_list[0].header["PIXSCAY"],
+            hdu_list[0].header["PIXSCAX"],
         )
-        pre_cti_data = aa.Array2D.from_primary_hdu(
-            primary_hdu=fit.value(name=f"{folder}.pre_cti_data")
+
+        mask = aa.Mask2D(
+            mask=ndarray_via_hdu_from(hdu_list[0]).astype("bool"),
+            pixel_scales=pixel_scales,
         )
-        try:
-            cosmic_ray_map = aa.Array2D.from_primary_hdu(
-                primary_hdu=fit.value(name=f"{folder}.cosmic_ray_map")
+
+        def values_from(hdu: int) -> aa.Array2D:
+            return aa.Array2D.no_mask(
+                values=ndarray_via_hdu_from(hdu_list[hdu]),
+                pixel_scales=pixel_scales,
             )
-        except AttributeError:
+
+        try:
+            cosmic_ray_map = values_from(hdu=4)
+        except IndexError:
             cosmic_ray_map = None
 
         settings_dict = fit.value(name="dataset.settings_dict")
 
         dataset = ImagingCI(
-            data=data,
-            noise_map=noise_map,
-            pre_cti_data=pre_cti_data,
+            data=values_from(hdu=1),
+            noise_map=values_from(hdu=2),
+            pre_cti_data=values_from(hdu=3),
             cosmic_ray_map=cosmic_ray_map,
             settings_dict=settings_dict,
             layout=layout,
         )
 
-        mask = aa.Mask2D.from_primary_hdu(primary_hdu=fit.value(name=f"{folder}.mask"))
-
         dataset_list.append(dataset.apply_mask(mask=mask))
 
     return dataset_list