diff --git a/markdown/interferometer/start_here.md b/markdown/interferometer/start_here.md index 87e6e8519..da57e8119 100644 --- a/markdown/interferometer/start_here.md +++ b/markdown/interferometer/start_here.md @@ -174,11 +174,25 @@ We begin by loading an `Interferometer` dataset from FITS, three ingredients are We must also choose a transformer for mapping the real-space image to visibilities: -- `TransformerNUFFT`: JAX-native Non-Uniform FFT (default, backed by `nufftax`). Recommended for any - dataset size — runs at full GPU speed for millions of visibilities. -- `TransformerDFT`: exact Discrete FT. Slower than the NUFFT for large `n_vis`, but useful as a reference - for verification and for the pixelized source reconstruction's sparse-operator workflow (see - `features/pixelization`). +- `TransformerNUFFT`: JAX-native Non-Uniform FFT (default, backed by `nufftax`). The right choice for + real datasets — runs at full GPU speed for millions of visibilities. +- `TransformerDFT`: exact Discrete FT, pure numpy. Useful as a reference for verification, and + genuinely faster than the NUFFT on small problems. + +Both are supported everywhere, including the pixelized source reconstruction's sparse-operator +workflow (`apply_sparse_operator`, see `features/pixelization`), where they agree to ~3e-13. + +Which is faster is set by the product `N_vis x N_pix`, not by the visibility count alone — the DFT +costs `O(N_vis x N_pix)` while the NUFFT costs `O((N_vis + N_pix) log N)` on top of a fixed ~2s +setup. Below `N_vis x N_pix ~ 1e7` the DFT wins; above it the NUFFT does. At a typical 64x64 mask +that crossover is near 5,000 visibilities, but on a coarser mask the DFT stays ahead well past that. + +Past `~1e8` the choice stops being about speed: the DFT's `O(N_vis x N_pix)` allocation reaches +~109 GB at a million visibilities, where the NUFFT allocates essentially nothing beyond its working +buffers. That is why real ALMA-scale work uses the NUFFT. Loading a dataset with more than 10,000 +visibilities and `TransformerDFT` raises an error for this reason; pass +`raise_error_dft_visibilities_limit=False` if you are deliberately doing a slow reference run. + We load a low resolution Square Mile Array (SMA) dataset for this example, which has just 273 visibilities. We use `TransformerNUFFT` so that this example reflects the recommended workflow at any visibility count; diff --git a/notebooks/interferometer/start_here.ipynb b/notebooks/interferometer/start_here.ipynb index 01c7c8b03..df6dda6a7 100644 --- a/notebooks/interferometer/start_here.ipynb +++ b/notebooks/interferometer/start_here.ipynb @@ -190,11 +190,24 @@ "\n", "We must also choose a transformer for mapping the real-space image to visibilities:\n", "\n", - "- `TransformerNUFFT`: JAX-native Non-Uniform FFT (default, backed by `nufftax`). Recommended for any\n", - " dataset size \u2014 runs at full GPU speed for millions of visibilities.\n", - "- `TransformerDFT`: exact Discrete FT. Slower than the NUFFT for large `n_vis`, but useful as a reference\n", - " for verification and for the pixelized source reconstruction's sparse-operator workflow (see\n", - " `features/pixelization`).\n", + "- `TransformerNUFFT`: JAX-native Non-Uniform FFT (default, backed by `nufftax`). The right choice for\n", + " real datasets \u2014 runs at full GPU speed for millions of visibilities.\n", + "- `TransformerDFT`: exact Discrete FT, pure numpy. Useful as a reference for verification, and\n", + " genuinely faster than the NUFFT on small problems.\n", + "\n", + "Both are supported everywhere, including the pixelized source reconstruction's sparse-operator\n", + "workflow (`apply_sparse_operator`, see `features/pixelization`), where they agree to ~3e-13.\n", + "\n", + "Which is faster is set by the product `N_vis x N_pix`, not by the visibility count alone \u2014 the DFT\n", + "costs `O(N_vis x N_pix)` while the NUFFT costs `O((N_vis + N_pix) log N)` on top of a fixed ~2s\n", + "setup. Below `N_vis x N_pix ~ 1e7` the DFT wins; above it the NUFFT does. At a typical 64x64 mask\n", + "that crossover is near 5,000 visibilities, but on a coarser mask the DFT stays ahead well past that.\n", + "\n", + "Past `~1e8` the choice stops being about speed: the DFT's `O(N_vis x N_pix)` allocation reaches\n", + "~109 GB at a million visibilities, where the NUFFT allocates essentially nothing beyond its working\n", + "buffers. That is why real ALMA-scale work uses the NUFFT. Loading a dataset with more than 10,000\n", + "visibilities and `TransformerDFT` raises an error for this reason; pass\n", + "`raise_error_dft_visibilities_limit=False` if you are deliberately doing a slow reference run.\n", "\n", "We load the ALMA long-baseline Science Verification observations of **SDP.81** \u2014 the famous\n", "z = 3.042 dusty star-forming galaxy lensed into an Einstein ring by a z = 0.299 foreground\n", diff --git a/scripts/interferometer/start_here.py b/scripts/interferometer/start_here.py index 70d5ed46c..57a9f323d 100644 --- a/scripts/interferometer/start_here.py +++ b/scripts/interferometer/start_here.py @@ -152,11 +152,24 @@ We must also choose a transformer for mapping the real-space image to visibilities: -- `TransformerNUFFT`: JAX-native Non-Uniform FFT (default, backed by `nufftax`). Recommended for any - dataset size — runs at full GPU speed for millions of visibilities. -- `TransformerDFT`: exact Discrete FT. Slower than the NUFFT for large `n_vis`, but useful as a reference - for verification and for the pixelized source reconstruction's sparse-operator workflow (see - `features/pixelization`). +- `TransformerNUFFT`: JAX-native Non-Uniform FFT (default, backed by `nufftax`). The right choice for + real datasets — runs at full GPU speed for millions of visibilities. +- `TransformerDFT`: exact Discrete FT, pure numpy. Useful as a reference for verification, and + genuinely faster than the NUFFT on small problems. + +Both are supported everywhere, including the pixelized source reconstruction's sparse-operator +workflow (`apply_sparse_operator`, see `features/pixelization`), where they agree to ~3e-13. + +Which is faster is set by the product `N_vis x N_pix`, not by the visibility count alone — the DFT +costs `O(N_vis x N_pix)` while the NUFFT costs `O((N_vis + N_pix) log N)` on top of a fixed ~2s +setup. Below `N_vis x N_pix ~ 1e7` the DFT wins; above it the NUFFT does. At a typical 64x64 mask +that crossover is near 5,000 visibilities, but on a coarser mask the DFT stays ahead well past that. + +Past `~1e8` the choice stops being about speed: the DFT's `O(N_vis x N_pix)` allocation reaches +~109 GB at a million visibilities, where the NUFFT allocates essentially nothing beyond its working +buffers. That is why real ALMA-scale work uses the NUFFT. Loading a dataset with more than 10,000 +visibilities and `TransformerDFT` raises an error for this reason; pass +`raise_error_dft_visibilities_limit=False` if you are deliberately doing a slow reference run. We load the ALMA long-baseline Science Verification observations of **SDP.81** — the famous z = 3.042 dusty star-forming galaxy lensed into an Einstein ring by a z = 0.299 foreground