MOC-based query by mcoughlin · Pull Request #401 · boom-astro/boom

mcoughlin · 2026-02-28T02:04:50Z

This PR adds a MOC-based query to supplement the cone search option

Copilot

Pull request overview

This PR adds support for MOC-based spatial alert searches (including HEALPix skymap thresholding) as an alternative/supplement to cone-search, along with supporting utilities, API wiring, and indexing.

Changes:

Introduces utils::moc helpers to parse MOC/skymap FITS, test containment, and build a cone-cover approximation for MongoDB geospatial queries.
Adds a new Babamul API endpoint POST /surveys/{survey}/alerts/moc-search and wires it into routing + OpenAPI.
Adds a MongoDB index on candidate.jd to speed up time-window filtering used by MOC searches.

Reviewed changes

Copilot reviewed 9 out of 10 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
`src/utils/mod.rs`	Exposes the new `moc` utility module.
`src/utils/moc.rs`	Implements MOC/skymap FITS parsing, containment, cone covering, and unit tests.
`src/utils/db.rs`	Adds `candidate.jd` index creation for alerts collections.
`src/enrichment/lsst.rs`	Refactors LSST footprint MOC loading/containment to use shared `utils::moc`.
`src/bin/api.rs`	Increases JSON payload size limit under `/babamul` and registers the new route.
`src/api/routes/babamul/surveys/mod.rs`	Re-exports `moc_search_alerts`.
`src/api/routes/babamul/surveys/alerts.rs`	Adds the MOC search endpoint implementation.
`src/api/docs.rs`	Includes the new endpoint in OpenAPI docs.
`data/glg_healpix_all_bn200524211.fits`	Adds a (LFS) skymap FITS fixture used by MOC tests.
`tests/data/alerts/ztf/ZTF20abbiixp_last_alert.json`	Adds a ZTF alert JSON fixture (currently unused).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

github-actions · 2026-02-28T02:17:28Z

Throughput results (e98e34723021b2bde87d46a42d02bde5bd4379b2):

New wall time	Baseline wall time	Difference
196.3	177.9	10.00%

github-actions · 2026-02-28T02:20:24Z

Throughput results (43a7ac92e41f85dbaf25956431ec08ed09e2829c):

New wall time	Baseline wall time	Difference
183.6	177.9	3.00%

github-actions · 2026-02-28T02:25:02Z

Throughput results (43e824a11cbaece830857c8fdff343e54d74cef7):

New wall time	Baseline wall time	Difference
185.5	177.9	4.00%

github-actions · 2026-02-28T02:25:16Z

Throughput results (269bef757ff5602c349fc765d8886fc5c991eeea):

New wall time	Baseline wall time	Difference
182.6	177.9	2.00%

github-actions · 2026-02-28T02:25:18Z

Throughput results (55e8dbb25e1556e651b6e31341721c3fb7ed65a0):

New wall time	Baseline wall time	Difference
182.6	177.9	2.00%

github-actions · 2026-02-28T02:25:55Z

Throughput results (85d1573b352d022df3aaf50ad8d245b29c73fbed):

New wall time	Baseline wall time	Difference
182.9	177.9	2.00%

github-actions · 2026-02-28T02:31:28Z

Throughput results (d296e8959fcf89861667478206451ebc287d1f7c):

New wall time	Baseline wall time	Difference
183.9	177.9	3.00%

Theodlz · 2026-02-28T08:25:36Z

+                    // Larger JSON payload limit for MOC/skymap uploads (100 MB)
+                    .app_data(web::JsonConfig::default().limit(104_857_600))


I've never seen a MOC that big????????? This change seems weird to me.

and even then, I would not want this parameter on all other endpoints

Theodlz · 2026-02-28T08:28:22Z

+    // create an index on candidate.jd for temporal range queries (MOC search, etc.)
+    let jd_index = doc! {
+        "candidate.jd": 1,
+    };
+    create_index(&alerts_collection, jd_index, false).await?;
+


we may want to just start adding some proper indexes on the parameters we care about. A query JUST on JD (MongoDB indexes should be compound as much as possible, since 2 different indexes aren't used together like they may in postgres) is very unlikely. I would suggest considering other fields we use on these queries.

I mean here for example I see you clearly run cone searches. So at minimum, having JD and coordinates is a must.

Theodlz · 2026-02-28T08:31:04Z

+#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, ToSchema)]
+struct AlertsMocSearchQuery {
+    /// Base64-encoded MOC FITS file (exactly one of moc_fits_base64 or skymap_fits_base64 required)
+    moc_fits_base64: Option<String>,
+    /// Base64-encoded HEALPix skymap FITS file
+    skymap_fits_base64: Option<String>,
+    /// Credible level for skymap thresholding (optional, defaults to 0.9 if omitted when skymap_fits_base64 is provided)
+    credible_level: Option<f64>,
+    /// Start of time window (required, Julian Date)
+    start_jd: f64,
+    /// End of time window (required, Julian Date, max 7 days after start_jd)
+    end_jd: f64,
+    min_magpsf: Option<f64>,
+    max_magpsf: Option<f64>,
+    #[serde(alias = "min_reliability")]
+    min_drb: Option<f64>,
+    #[serde(alias = "max_reliability")]
+    max_drb: Option<f64>,
+    is_rock: Option<bool>,
+    is_star: Option<bool>,
+    is_near_brightstar: Option<bool>,
+    is_stationary: Option<bool>,


To echo my other comment on the index: why not just have a compound index on all these parameters so we know for SURE that query is covered???

github-actions · 2026-02-28T11:37:41Z

Throughput results (523230ee8926326d73eae49f155281e8bcd8fe91):

New wall time	Baseline wall time	Difference
180.3	177.9	1.00%

github-actions · 2026-05-30T08:58:49Z

Throughput results (b04332b07c9d7273ae5e19077a1152a02e385513):

Storage	New wall time	Baseline wall time	Difference
mongo	130.6	192.6	-32.00%
s3	233.9	294.1	-20.00%

github-actions · 2026-05-30T21:41:29Z

Throughput results (2851e7c2080145dba953a93bb4f005868d3d22fd):

Storage	New wall time	Baseline wall time	Difference
mongo	250.0	192.6	29.00%
s3	233.9	294.1	-20.00%

github-actions · 2026-06-01T04:07:11Z

Throughput results (91d0c3775d00bf20d3c5fa8d3b66288940b17e5e):

Storage	New wall time	Baseline wall time	Difference
mongo	192.0	192.6	0%
s3	294.1	294.1	0%

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…by survey.

…inates.

github-actions · 2026-06-16T17:39:47Z

Throughput results (42f2d18c08213b97c0a0edeb892ba66a0fc65228):

Storage	New wall time	Baseline wall time	Difference
mongo	245.3	234.1	4.00%
s3	274.4	294.1	-6.00%

Baseline run: 27294071934

github-actions · 2026-06-17T20:36:36Z

Throughput results (863528eb0435d70bf19c1a2dacec66141f82bd18):

Storage	New wall time	Baseline wall time	Difference
mongo	250.6	234.1	7.00%
s3	294.3	294.1	0%

Baseline run: 27294071934

github-actions · 2026-06-18T13:56:30Z

Throughput results (c236db3d447a9f14e14bff96b95c7c0d8f02adf5):

Storage	New wall time	Baseline wall time	Difference
mongo	253.4	234.1	8.00%
s3	294.1	294.1	0%

Baseline run: 27294071934

Copilot AI review requested due to automatic review settings February 28, 2026 02:04

Copilot started reviewing on behalf of mcoughlin February 28, 2026 02:05 View session

Copilot AI reviewed Feb 28, 2026

View reviewed changes

mcoughlin requested a review from antoine-le-calloch February 28, 2026 02:31

mcoughlin requested a review from Theodlz February 28, 2026 02:31

Theodlz reviewed Feb 28, 2026

View reviewed changes

antoine-le-calloch assigned mcoughlin, frenbox and antoine-le-calloch and unassigned mcoughlin and antoine-le-calloch Jun 9, 2026

mcoughlin and others added 6 commits June 16, 2026 11:21

Add MOC-based query

2f2e5ce

pre-commit

385983f

Update src/utils/moc.rs

b6c0145

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/api/routes/babamul/surveys/alerts.rs

034c2c9

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/api/routes/babamul/surveys/alerts.rs

4f88b9f

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/api/routes/babamul/surveys/alerts.rs

92c0c2f

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

mcoughlin and others added 8 commits June 16, 2026 11:21

copilot comments

dd995e2

Theo's comments

4e258c0

Offload MOC parsing and cone computation to blocking thread pool.

d7238fe

Validate credible_level for MOC queries to prevent misuse.

53c3824

Refactor MOC alert retrieval into a reusable function to deduplicate …

88407d6

…by survey.

Deduplicate ZTF/LSST branches in cone search via cone_search_by_coord…

655c53f

…inates.

Fix endpoints order.

12e89de

enabling 3d skymap crossmatches

1e643c5

frenbox force-pushed the add-moc-spatial-query branch from 81506d8 to 1e643c5 Compare June 16, 2026 16:33

minor changes in Docerfile

1bb2dd3

antoine-le-calloch removed their request for review June 17, 2026 14:08

frenbox added 2 commits June 17, 2026 15:08

minor changes

3f728bf

adding new catalog crossmatches

b8d3653

frenbox added 2 commits June 17, 2026 19:40

API changes

15da71f

minor changes in credible interval

095979f

		// Larger JSON payload limit for MOC/skymap uploads (100 MB)
		.app_data(web::JsonConfig::default().limit(104_857_600))

Uh oh!

Conversation

mcoughlin commented Feb 28, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Feb 28, 2026

Uh oh!

github-actions Bot commented Feb 28, 2026

Uh oh!

github-actions Bot commented Feb 28, 2026

Uh oh!

github-actions Bot commented Feb 28, 2026

Uh oh!

github-actions Bot commented Feb 28, 2026

Uh oh!

github-actions Bot commented Feb 28, 2026

Uh oh!

github-actions Bot commented Feb 28, 2026

Uh oh!

Theodlz Feb 28, 2026

Choose a reason for hiding this comment

Uh oh!

Theodlz Feb 28, 2026

Choose a reason for hiding this comment

Uh oh!

Theodlz Feb 28, 2026

Choose a reason for hiding this comment

Uh oh!

Theodlz Feb 28, 2026

Choose a reason for hiding this comment

Uh oh!

Theodlz Feb 28, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Feb 28, 2026

Uh oh!

github-actions Bot commented May 30, 2026

Uh oh!

github-actions Bot commented May 30, 2026

Uh oh!

github-actions Bot commented Jun 1, 2026

Uh oh!

github-actions Bot commented Jun 16, 2026

Uh oh!

github-actions Bot commented Jun 17, 2026

Uh oh!

github-actions Bot commented Jun 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants