@@ -310,11 +310,13 @@ the full list and dataset notes.
310310## Results
311311
312312Correctness is the headline, but every case is also profiled. The numbers below
313- come from [ ` run_perf.sh ` ] ( ../benchmarks/geospatial/run_perf.sh ) on a Google Compute
314- Engine ` e2-standard-8 ` (8 vCPU, 32 GB) in ` us-central1 ` — in-region with the
315- ARCO-ERA5 and WeatherBench 2 buckets, so the cloud read is fast. Each case runs
316- ** once per fresh process** , with no warmup, repeated five times: the SQL operation
317- * and* the xarray reference each pay a ** cold** read on every measurement.
313+ come from [ ` run_perf.sh ` ] ( ../benchmarks/geospatial/run_perf.sh ) on a single Google
314+ Compute Engine ` e2-standard-8 ` (8 vCPU, 32 GB) in ` us-central1 ` — in-region with the
315+ ARCO-ERA5 and WeatherBench 2 buckets, so the cloud read is fast — with Earth Engine
316+ reached from the same VM, so all nine cases share one machine and one release build.
317+ Each case runs ** once per fresh process** , with no warmup, repeated five times: the
318+ SQL operation * and* the xarray reference each pay a ** cold** read on every
319+ measurement.
318320
319321Fairness here took some care, because the obvious trap is caching. A reference
320322that calls ` .load() ` caches its data * in place* on the very object the SQL table
@@ -331,45 +333,56 @@ speeds up neither — the SQL query and the reference do not warm each other.
331333
332334| Case | Step | median (s) | stdev (s) | min (s) | max (s) | peak (MB) |
333335| ---| ---| --:| --:| --:| --:| --:|
334- | 01 · NDVI (per-pixel arithmetic) | SQL | 3.093 | 0.050 | 2.998 | 3.128 | 98.5 |
335- | | xarray reference | 0.402 | 0.060 | 0.372 | 0.507 | 42.0 |
336- | 02 · Climatology (` GROUP BY ` lat, lon, hour) | SQL | 5.987 | 1.328 | 5.901 | 8.942 | 513.8 |
337- | | xarray reference | 2.449 | 0.401 | 2.332 | 3.312 | 43.7 |
338- | 03 · Zonal mean (` GROUP BY ` latitude) | SQL | 3.224 | 0.048 | 3.175 | 3.288 | 245.3 |
339- | | xarray reference | 0.552 | 0.020 | 0.529 | 0.576 | 249.5 |
340- | 04 · Anomaly (climatology self-` JOIN ` ) | SQL | 9.479 | 0.413 | 9.382 | 10.280 | 524.4 |
341- | | xarray reference | 3.013 | 0.584 | 2.875 | 4.199 | 80.5 |
342- | 05 · Forecast skill (forecast↔truth ` JOIN ` ) | SQL | 23.241 | 0.101 | 23.114 | 23.348 | 34.2 |
343- | | xarray reference | 0.336 | 0.014 | 0.314 | 0.352 | 2.2 |
344- | 06 · Zonal stats (raster × vector ` JOIN ` ) | SQL | 6.285 | 0.052 | 6.191 | 6.317 | 509.9 |
345- | | xarray reference | 2.235 | 0.216 | 2.195 | 2.703 | 359.9 |
346- | 07 · Reprojection (PROJ scalar UDF) | SQL | 0.039 | 0.000 | 0.039 | 0.039 | 0.3 |
347- | 08 · Regridding (weight-table ` JOIN ` ) | SQL | 0.061 | 0.001 | 0.060 | 0.063 | 0.8 |
348- | | xarray reference | 0.018 | 0.001 | 0.018 | 0.020 | 0.2 |
349-
350- Two patterns are visible before any analysis. SQL is slower on wall-clock in every
351- case — by ~ 2–6× on the plain ` GROUP BY ` s, and ~ 70× on case 05, the smallest grid
352- but the largest ` JOIN ` — and its peak memory is markedly higher on the
353- join/group-by cases (≈0.5 GB on 02, 04, 06). Both follow from the same cause, and
354- the next section pins it down. (Case 01 reads Sentinel-2 from Europe, the only
355- non-US source, so its SQL time includes a cross-region read. Cases 07–09 load their
356- Earth Engine inputs into memory once and then compute, so they are
357- methodology-agnostic; cases 07 and 09 time only the SQL transform — checked against
358- Earth Engine's own ` pixelLonLat ` /SRTM — and run ` reps=1 ` because PROJ is not
359- re-entrant in-process.)
360-
361- Case 05 is also the suite's most hardware-sensitive number: its SQL time is
362- CPU-bound on the join and the (GIL-held) row production that feeds it, so it swings
363- with the machine — a second ` e2-standard-8 ` run measured ≈12 s rather than ≈23 s.
364- The * reference* , by contrast, is read-bound and stable. So read the 05 ratio as
336+ | 01 · NDVI (per-pixel arithmetic) | SQL | 3.528 | 0.803 | 2.861 | 5.024 | 114.0 |
337+ | | xarray reference | 0.304 | 0.104 | 0.282 | 0.496 | 42.0 |
338+ | 02 · Climatology (` GROUP BY ` lat, lon, hour) | SQL | 4.443 | 0.383 | 4.216 | 5.198 | 490.2 |
339+ | | xarray reference | 1.867 | 0.106 | 1.844 | 2.053 | 43.7 |
340+ | 03 · Zonal mean (` GROUP BY ` latitude) | SQL | 2.406 | 0.122 | 2.333 | 2.631 | 236.9 |
341+ | | xarray reference | 0.385 | 0.006 | 0.381 | 0.395 | 249.5 |
342+ | 04 · Anomaly (climatology self-` JOIN ` ) | SQL | 7.027 | 0.123 | 6.950 | 7.239 | 511.5 |
343+ | | xarray reference | 2.549 | 0.219 | 2.126 | 2.657 | 72.1 |
344+ | 05 · Forecast skill (forecast↔truth ` JOIN ` ) | SQL | 10.714 | 0.093 | 10.663 | 10.891 | 6.6 |
345+ | | xarray reference | 0.248 | 0.013 | 0.220 | 0.254 | 2.2 |
346+ | 06 · Zonal stats (raster × vector ` JOIN ` ) | SQL | 4.308 | 0.053 | 4.299 | 4.401 | 509.1 |
347+ | | xarray reference | 1.557 | 0.029 | 1.499 | 1.567 | 1262.1 |
348+ | 07 · Reprojection (PROJ scalar UDF) | SQL | 0.029 | 0.003 | 0.024 | 0.031 | 0.3 |
349+ | 08 · Regridding (weight-table ` JOIN ` ) | SQL | 0.875 | 0.037 | 0.845 | 0.933 | 11.9 |
350+ | | xarray reference | 0.850 | 0.658 | 0.809 | 2.310 | 13.3 |
351+ | 09 · Warp (reproject UDF → regrid ` JOIN ` ) | SQL | 0.281 | 0.038 | 0.250 | 0.353 | 0.8 |
352+ | | xarray reference | 0.817 | 0.030 | 0.764 | 0.828 | 11.2 |
353+
354+ Two patterns are visible before any analysis. SQL is slower on wall-clock wherever
355+ a cloud read or a large relational expansion dominates — by ~ 2.5–6× on the
356+ ` GROUP BY ` and ` JOIN ` cases against ARCO-ERA5, and ~ 43× on case 05, the smallest
357+ grid but the biggest blow-up into rows — and its peak memory is highest on those
358+ join/group-by cases (≈0.5 GB on 02, 04, 06). But the pattern is ** not** universal.
359+ On cases 08 and 09, where the interpolation * weights* are precomputed and SQL just
360+ applies them, SQL is at parity with the array reference (08: 0.875 vs 0.850 s) or
361+ ** faster** (09: 0.281 vs 0.817 s — the reference pays for ` pyproj ` + ` .interp ` ,
362+ while SQL streams the prebuilt weight ` JOIN ` ). The slow and the fast cases follow
363+ from the same cause, which the next section pins down. (Case 01 reads Sentinel-2
364+ from Europe, the only non-US source, so its SQL time includes a cross-region read.
365+ Cases 07–09 run against Earth Engine from the same VM: 07 times only the SQL
366+ reproject transform, checked against Earth Engine's own ` pixelLonLat ` ; 08 and 09
367+ read SRTM lazily on ** both** the SQL and reference sides, so that comparison is
368+ symmetric.)
369+
370+ Case 05 is the suite's most hardware-sensitive number: its SQL time is CPU-bound on
371+ the join and the (GIL-held) row production that feeds it, so it swings with the
372+ machine — across three ` e2-standard-8 ` runs it has measured ≈10.7 s, ≈12 s, and
373+ ≈23 s, while the read-bound * reference* stays near 0.25 s. So read the 05 ratio as
365374"the relational form costs real CPU here," not as a fixed multiplier.
366375
367376## Analysis: how a relational operation spends its time
368377
369378Why is SQL slower, and where does the time actually go? Profiling case 05 — the
370- forecast-skill ` JOIN ` , the widest gap — with ` cProfile ` , one path per fresh
371- process, run cold then warm so that ` cold − warm ` isolates the cloud read and the
372- warm floor is ≈pure compute, decomposes it cleanly:
379+ forecast-skill ` JOIN ` , the widest gap — with ` cProfile ` , run cold then warm so that
380+ ` cold − warm ` isolates the cloud read and the warm floor is ≈pure compute,
381+ decomposes it cleanly. (These are single-process numbers from a laptop with a slow
382+ cross-region read — a * different* machine from the in-region table above, on
383+ purpose: it puts both sides' reads on equal, slow footing so the compute gap shows
384+ through. The absolute seconds therefore differ from the table; the decomposition,
385+ not the totals, is the point.)
373386
374387| | read (I/O) | compute | total (cold) |
375388| ---| --:| --:| --:|
@@ -393,30 +406,42 @@ round-trip that turns the query result back into a gridded `Dataset`
393406relational engine doing row-oriented work, not the array reconstruction. The
394407paradigm itself is the price, paid where the relational algebra runs.
395408
396- This explains the shape of the whole table. The ` JOIN ` cases (04, 05, 06) show the
397- widest gaps because a hash join is the heaviest relational construct; the plain
398- ` GROUP BY ` cases (02, 03) are closest to parity because a partitioned aggregate is
399- cheap. And the SQL-to-reference * ratio* shifts with hardware: SQL is CPU-bound on
400- the join, while the array reference is read-bound, so the two are gated by
401- different resources. On a fast laptop with a slow cross-region read the gap nearly
402- closes; on an in-region VM with modest cores it widens. The underlying cause is
409+ This explains the shape of the whole table. Case 05 stands alone at ~ 43× not
410+ because its join is exotic but because its * reference* is nearly free — a 64×32×20×2
411+ grid reduces in-memory in a quarter-second — while SQL still has to explode that
412+ grid into rows and hash-join them; a huge ratio over a tiny denominator. The
413+ ARCO-ERA5 cases (02, 03, 04, 06) instead cluster at ~ 2.5–6×, because there a large
414+ cloud read is a cost * both* sides pay, compressing the ratio. And cases 08 and 09
415+ invert it entirely: once the geometry — the interpolation weights — is precomputed,
416+ applying it is a ` JOIN ` that streams about as fast as (or faster than) the array
417+ reference's ` pyproj ` /` .interp ` . The relational * overhead* is constant; the * ratio*
418+ you observe depends on how much non-relational work (the cloud read, the weight
419+ generation) sits on the other side of the comparison. And it shifts with hardware
420+ too: SQL is CPU-bound on the join while the array reference is read-bound, so the
421+ two are gated by different resources. On a fast laptop with a slow cross-region
422+ read the gap nearly closes; on an in-region VM with modest cores it widens. The
423+ underlying cause is
403424constant — materialize rows, hash-join, aggregate — but which resource you are
404425waiting on is not.
405426
406427## Conclusion
407428
408- None of this is an argument that SQL is * faster* . On a single node, for these
409- operations, it is not — it pays a real per-operation overhead to express an array
410- reduction as relational algebra. The honest tradeoff is about which property you
411- are optimizing for.
429+ None of this is an argument that SQL is * faster* . On a single node, for the
430+ reduction-shaped operations, it is not — it pays a real per-operation overhead to
431+ express an array reduction as relational algebra. (The exceptions, cases 08 and 09,
432+ prove the rule: once the array work — generating the weights — is already done, the
433+ relational half that remains is competitive, because there is no dense reduction
434+ left for arrays to win.) The honest tradeoff is about which property you are
435+ optimizing for.
412436
413437** Reach for the array paradigm when the work is dense and grid-aligned.** Per-pixel
414438formulas, stencils, convolutions, FFTs, linear algebra — anything that stays in
415439contiguous typed buffers and treats the chunk grid as its unit of parallelism. The
416440array model has the lowest overhead here, and the lead is structural, not
417441incidental: there are no rows to materialize and nothing to shuffle. NDVI (case 01)
418442is the tell — column arithmetic expresses cleanly in SQL, but the array side is
419- ~ 8× faster because per-pixel math is exactly what arrays are for.
443+ ~ 10× faster (part of which is case 01's cross-region read; the rest is that
444+ per-pixel math is exactly what arrays are for).
420445
421446** Reach for SQL when the work is relationally shaped, or the audience is.** Joins,
422447group-bys, alignment across data with different indexes (case 05's three time
0 commit comments