Impure mass distribution exact [version:1.0.1] #66

henriquelettieri · 2025-12-10T12:43:47Z

Implementation of 3D inegral for impure cluster counts

eduardojsbarroso · 2025-12-11T08:15:32Z

crow/recipes/binned_exact.py

-        """Makes mass distribution use additional integral with completeness"""
-        if self.purity is None:
+        """Makes mass distribution use additional integral with purity"""
+        if self.purity is None: 


I think this is good. Because if there is purity, we have to change both the mass distribution and the purity function. I just don't think it is a good name to say then that the mass distribtuion impure is mass_dist * purity. But just a naming problem

eduardojsbarroso · 2025-12-11T08:16:10Z

crow/recipes/binned_exact.py

            self._mass_distribution_distribution = self._impure_mass_distribution

-    def _impure_mass_distribution(self, log_mass, z, log_mass_proxy_limits):
+    def _impure_mass_distribution(self, log_mass, z, log_mass_proxy):


This here is the big problem for all the functions right? Because the impure does not take only a tuple but an NDArray of log_mass_proxy

eduardojsbarroso · 2025-12-11T08:24:48Z

crow/recipes/binned_exact.py


        return theory_prediction

+    def _get_theory_prediction_impure_counts(


It is right, it works, but I do not like this. Because we are repeating so much code with all these functions. And if you go with these option, you will also have to duplicate everything for shear. My solution:

You make a get theory_prediction_counts that returns a callable like the impure one:

Callable[ [npt.NDArray[np.float64], npt.NDArray[np.float64], npt.NDArray[np.float64], float], npt.NDArray[np.float64],

However:

prediction = ( self.cluster_theory.comoving_volume(z, sky_area) * self.cluster_theory.mass_function(mass, z) * self._completeness_distribution(mass, z) * self.redshift_distribution.distribution() ) If purity is none: assert(mass_proxy.dim == 2) # massproxy is an NDArray with 2 values prediction *= self._mass_distribution_distribution(mass, z, (mass_proxy[0], mass_proxy[1])) else: call with the NDarray

eduardojsbarroso · 2025-12-11T08:25:46Z

crow/recipes/binned_exact.py

+        ) -> npt.NDArray[np.float64]:
+            mass = int_args[:, 0]
+            z = int_args[:, 1]
+            mass_proxy = int_args[:, 2]


Then here you check again if there is purity or not. If there is,mass_proxy is like this, an int arg. If there isnt, mass proxy is an extra arg, but continue to be an NDArray

eduardojsbarroso · 2025-12-11T08:29:13Z

@henriquelettieri If you apply the changes I did, you will only need one function for counts and one for shear.

codecov · 2025-12-12T18:12:35Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.36%. Comparing base (a7b4813) to head (efbd70d).

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #66      +/-   ##
==========================================
+ Coverage   98.27%   98.36%   +0.09%     
==========================================
  Files          28       28              
  Lines        1677     1770      +93     
  Branches       51       59       +8     
==========================================
+ Hits         1648     1741      +93     
  Misses         29       29

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

eduardojsbarroso · 2025-12-15T08:19:04Z

crow/recipes/binned_exact.py

-        ]
-        self.integrator.extra_args = np.array([*log_proxy_edges, sky_area])
+        assert len(log_proxy_edges) == 2
+        assert len(z_edges) == 2


This test should be in get_theorey_prediction and not here. Even if here things are 2d, if the integrator calls the function get_theory_prediction with these intervals, it will give a NDArray with several values, and if purity was not properly set, this can be a problem.

eduardojsbarroso · 2025-12-15T08:20:16Z

@henriquelettieri It seems good with this approach. Nice job. Please change the test where it makes more sense, then do the same to shear profile and add tests

…alculations

eduardojsbarroso · 2026-01-05T09:37:36Z

@henriquelettieri is this ready? If so, could you make sure it passes all the tests so I can review?

henriquelettieri · 2026-01-05T15:45:41Z

@eduardojsbarroso In principle everything is ready, I may add new tests. However the exact purity calculation for the shear is way too slow when using the same method for counts. I will fix the tests today

…nd create test_evaluates_theory_prediction_assertions

eduardojsbarroso

Besides the change in how to call the compute with shear, it seems good. However the 10% reldif tells me something is wrong. Also we have to decide if the purity will indeed be included in the shear as I said on slack. @henriquelettieri

eduardojsbarroso · 2026-01-12T10:17:19Z

crow/recipes/binned_exact.py

+            deltasigma_list = []
+            for radius_center in radius_centers:
+                self.integrator.extra_args = np.array(
+                    [*log_proxy_edges, sky_area, radius_center]


I think we can change here so we avoid repeating the code:

if self.purity == None: self.integrator.integral_bounds = [ self.mass_interval, z_edges, ] extra_args = [*log_proxy_edges] else: if self.purity == None: self.integrator.integral_bounds = [ self.mass_interval, z_edges, log_proxy_edges, ] extra_args = [] for radius_center in radius_centers: self.integrator.extra_args = np.array( extra_args.extend([sky_area, radius_center]))

Then we dont need to rewrite everything for each case

eduardojsbarroso · 2026-01-12T10:19:22Z

notebooks/grid_new.ipynb

+   "version": "3.14.1"
  }
 },
 "nbformat": 4,


If you did not add anything to this notebook, just ran it, please restore it before commiting

eduardojsbarroso · 2026-01-12T10:57:00Z

tests/test_recipe_binned_counts.py

        z_array = np.array([z_scalar])
        log_proxy_array = np.array([log_proxy_scalar])
-        return binned_grid_w_pur._purity_distribution(log_proxy_array, z_array)
+        return binned_grid_w_pur._purity_distribution(log_proxy_array, z_array).item()


Why did you change this here? We did not change the _purity_distribution function in this PR right? I get it that quad will call this with scalars and we want to return a scalar, but was this not working before?

eduardojsbarroso · 2026-01-12T13:04:43Z

tests/test_recipe_binned_shear_profile.py

    denom = np.where(pred_exact == 0, 1.0, pred_exact)
    assert np.all(np.abs((pred_grid - pred_exact) / denom) < rel_tol)

+    rel_tol = 1.0e-1


I think this is to low as relative tolerance. Because in the test with counts, we are able to test this with rel_tol = e-4. There must be either some parameter wrong here or something else.

I tested calling the completeness distrubtion before and after you change the property and it seems to be working fine, but we should investigate this. 10% while the counts works fine, and we can also have e-4 with purity, there must be something wrong here

henriquelettieri added 2 commits December 5, 2025 10:53

vectorized mass_distribution purity exact first version

963440d

recipe updated with 3d calculation for impure cluster counts

239d70b

eduardojsbarroso requested changes Dec 11, 2025

View reviewed changes

henriquelettieri added 3 commits December 11, 2025 08:02

combined purity non purity in one function

74fbc1f

binned_counts tests for new counts with purity code

96fc85d

binned_counts tests for new counts with purity code

47304c3

eduardojsbarroso reviewed Dec 15, 2025

View reviewed changes

henriquelettieri added 4 commits December 27, 2025 11:07

add new assert in _get_theory_prediction_counts and purity in shear c…

ec3bf74

…alculations

add new assert in _get_theory_prediction_counts and purity in shear c…

abcffb2

…alculations

add new assert in _get_theory_prediction_counts and purity in shear c…

4716996

…alculations

add new assert in _get_theory_prediction_counts and purity in shear c…

bf27930

…alculations

henriquelettieri added 6 commits January 5, 2026 10:23

added comparisons in test_grid_shear_matches_exact_within_tolerance a…

9e06d7c

…nd create test_evaluates_theory_prediction_assertions

added comparisons in test_grid_shear_matches_exact_within_tolerance a…

02a616e

…nd create test_evaluates_theory_prediction_assertions

added comparisons in test_grid_shear_matches_exact_within_tolerance a…

9c2101b

…nd create test_evaluates_theory_prediction_assertions

added comparisons in test_grid_shear_matches_exact_within_tolerance a…

9150e09

…nd create test_evaluates_theory_prediction_assertions

added comparisons in test_grid_shear_matches_exact_within_tolerance a…

04644b7

…nd create test_evaluates_theory_prediction_assertions

new version

a5af570

github-actions bot changed the title ~~Impure mass distribution exact~~ Impure mass distribution exact [version:1.0.1] Jan 5, 2026

new version

efbd70d

eduardojsbarroso reviewed Jan 12, 2026

View reviewed changes


		return theory_prediction

		def _get_theory_prediction_impure_counts(

Impure mass distribution exact [version:1.0.1] #66

Are you sure you want to change the base?

Impure mass distribution exact [version:1.0.1] #66

Conversation

henriquelettieri commented Dec 10, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

eduardojsbarroso commented Dec 11, 2025

Uh oh!

codecov bot commented Dec 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

eduardojsbarroso commented Dec 15, 2025

Uh oh!

eduardojsbarroso commented Jan 5, 2026

Uh oh!

henriquelettieri commented Jan 5, 2026

Uh oh!

eduardojsbarroso left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov bot commented Dec 12, 2025 •

edited

Loading

eduardojsbarroso left a comment •

edited

Loading