Commit 8cdfad5
committed
Document the DFT/NUFFT trade-off on the sparse-operator setup path
Both transformers work with `apply_sparse_operator` and agree to ~3e-13
relative, but which is *faster* was undocumented, and the answer is not the
one the codebase's existing 10,000-visibility guard implies.
Measured on CPU (`use_jax=False`), the crossover is governed by the product
`N_vis * N_pix`, not by visibility count alone — which follows from the DFT
setup being `O(N_vis * N_pix)` against the NUFFT's `O((N_vis + N_pix) log N)`
plus a fixed ~2s overhead. Seven measurements agree on a crossover near 1e7:
N_vis x N_pix DFT/NUFFT time ratio
1.3e6 0.21x (DFT faster)
2.6e6 0.27x
5.2e6 0.70x
1.0e7 1.16x (crossover)
2.1e7 1.50x
2.3e7 1.41x
8.3e7 1.92x (NUFFT faster)
So at a typical 64x64 mask the crossover sits near 5,000 visibilities, but on
a 32x32 mask the DFT still wins at 4,000. Quoting a visibility count alone
would be wrong on half the grids.
Memory is the more decisive axis and is what makes this more than a
micro-optimisation. The DFT's allocation grows with `N_vis * N_pix` (measured
60 -> 239 -> 293 -> 446 MB as the grid grows at fixed N_vis=4000) while the
NUFFT path allocates nothing measurable beyond its working buffers. At 10.7
bytes per element that extrapolates to ~109 GB at 1M visibilities, which
independently corroborates the ~123 GB figure recorded in bd18a76 and
confirms the NUFFT is the only feasible path at ALMA scale.
Adds the guidance as an INFO on the existing setup log, emitted only when a
NUFFT-backed transformer is used below the crossover — i.e. only on an
explicit, expensive, opt-in call, and never when the advice would be moot.
Deliberately NOT a warning on `Interferometer` construction: that class
defaults to `TransformerNUFFT`, so a small-dataset warning would fire on the
default path for every tutorial, test and workspace example. The dangerous
direction is already a hard `DatasetException` above 10,000 visibilities with
an opt-out, so nothing is left unguarded by keeping this advisory quiet.
Verified the advisory fires for small+NUFFT and stays silent for both
small+DFT and large+NUFFT. Suite green: 1179 passed.1 parent 46fc1c5 commit 8cdfad5
1 file changed
Lines changed: 29 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
232 | 232 | | |
233 | 233 | | |
234 | 234 | | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
235 | 250 | | |
236 | 251 | | |
237 | 252 | | |
| |||
264 | 279 | | |
265 | 280 | | |
266 | 281 | | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
267 | 296 | | |
268 | 297 | | |
269 | 298 | | |
| |||
0 commit comments