Expand margin generation explanation. Use anon references. (#485)

Author: delucchi-cmu

docs/catalogs/arguments.rst

@@ -5,7 +5,7 @@ This page discusses a few topics around setting up a catalog pipeline.
 
 Once your catalog has been imported, you can verify and inspect the metadata, 
 either with the :doc:`Verification Pipeline </guide/verification>`, or methods
-described in the `Manual catalog verification notebook <https://docs.lsdb.io/en/stable/tutorials/manual_verification.html>`_
+described in the `Manual catalog verification notebook <https://docs.lsdb.io/en/stable/tutorials/manual_verification.html>`__
 
 At a minimum, you need arguments that include where to find the input files,
 the column names for RA, and DEC, and where to put the output files. 

docs/guide/contact.rst

@@ -11,10 +11,10 @@ making our products better, or pretty much anything else, reach out!
 * Start a new github discussion
     * https://github.com/astronomy-commons/hats-import/discussions/new/choose
 * If you're on LSSTC slack, so are we!
-    * `#lincc-frameworks-qa <https://lsstc.slack.com/archives/C062LG1AK1S>`_
-    * `#lincc-frameworks-lsdb <https://lsstc.slack.com/archives/C04610PQW9F>`_
+    * `#lincc-frameworks-qa <https://lsstc.slack.com/archives/C062LG1AK1S>`__
+    * `#lincc-frameworks-lsdb <https://lsstc.slack.com/archives/C04610PQW9F>`__
 * Join the working group, where we discuss HATS standardization and early results
-    * Google group: `[email protected] <https://groups.google.com/g/hipscat-wg>`_
+    * Google group: `[email protected] <https://groups.google.com/g/hipscat-wg>`__
     * You can listen in to demo meetings, or ask questions during co-working sessions. 
       Events are published on a google calendar, embedded below.
     * Key:
@@ -34,4 +34,4 @@ making our products better, or pretty much anything else, reach out!
 
 However you reach out, we want to make sure that any discourse is open and 
 inclusive, and we ask that everyone involved read and adhere to the 
-`LINCC Frameworks Code of Conduct <https://lsstdiscoveryalliance.org/programs/lincc-frameworks/code-conduct/>`_
+`LINCC Frameworks Code of Conduct <https://lsstdiscoveryalliance.org/programs/lincc-frameworks/code-conduct/>`__

docs/guide/contributing.rst

@@ -5,7 +5,7 @@ HATS, hats-import, and LSDB are primarily written and maintained by LINCC Framew
 would love to turn it over to the open-source scientific community!! We want to 
 make sure that any discourse is open and inclusive, and we ask that everyone
 involved read and adhere to the 
-`LINCC Frameworks Code of Conduct <https://lsstdiscoveryalliance.org/programs/lincc-frameworks/code-conduct/>`_
+`LINCC Frameworks Code of Conduct <https://lsstdiscoveryalliance.org/programs/lincc-frameworks/code-conduct/>`__
 
 Installation from Source
 ------------------------
@@ -47,21 +47,21 @@ by creating a new GitHub issue.
 Note that you may need to make changes in multiple repos to fully implement new
 features or bug fixes! See related projects:
 
-* HATS (`on GitHub <https://github.com/astronomy-commons/hats>`_ 
-  and `on ReadTheDocs <https://hats.readthedocs.io/en/stable/>`_)
-* LSDB (`on GitHub <https://github.com/astronomy-commons/lsdb>`_
-  and `on ReadTheDocs <https://docs.lsdb.io>`_)
+* HATS (`on GitHub <https://github.com/astronomy-commons/hats>`__ 
+  and `on ReadTheDocs <https://hats.readthedocs.io/en/stable/>`__)
+* LSDB (`on GitHub <https://github.com/astronomy-commons/lsdb>`__
+  and `on ReadTheDocs <https://docs.lsdb.io>`__)
 
 Fork the repository
 -------------------------------------------------------------------------------
 
-Contributing to hats-import requires you to `fork <https://github.com/astronomy-commons/hats-import/fork>`_ 
+Contributing to hats-import requires you to `fork <https://github.com/astronomy-commons/hats-import/fork>`__ 
 the GitHub repository. The next steps assume the creation of branches and PRs are performed from your fork.
 
 .. note::
 
     If you are (or expect to be) a frequent contributor, you should consider requesting
-    access to the `hats-friends <https://github.com/orgs/astronomy-commons/teams/hats-friends>`_
+    access to the `hats-friends <https://github.com/orgs/astronomy-commons/teams/hats-friends>`__
     working group. Members of this GitHub group should be able to create branches and PRs directly
     on LSDB, hats and hats-import, without the need of a fork.
 
@@ -70,7 +70,7 @@ Testing
 
 We use ``pytest`` as our preferred unit test runner engine, in keeping with
 LSST DM style guide. We make heavy use of 
-`pytest fixtures <https://docs.pytest.org/en/7.1.x/explanation/fixtures.html#about-fixtures>`_, 
+`pytest fixtures <https://docs.pytest.org/en/7.1.x/explanation/fixtures.html#about-fixtures>`__, 
 which set up various resources used for unit testing, or provide consistent 
 paths. These are defined in ``conftest.py`` files. They're powerful and flexible 
 (and fun in their own way), and we encourage contributors to familiarize themselves.
@@ -126,4 +126,4 @@ Release new version
 
 New versions are manually tagged and automatically released to pypi. To request
 a new release of LSDB, HATS, and hats-import packages, create a 
-`release ticket <https://github.com/astronomy-commons/lsdb/issues/new?assignees=delucchi-cmu&labels=&projects=&template=4-release_tracker.md&title=Release%3A+>`_.
+`release ticket <https://github.com/astronomy-commons/lsdb/issues/new?assignees=delucchi-cmu&labels=&projects=&template=4-release_tracker.md&title=Release%3A+>`__.

docs/guide/hipscat_conversion.rst

@@ -129,4 +129,4 @@ What next?
 
 You can validate that your new HATS catalog meets both the HATS/LSDB expectations,
 as well as your own expectations of the data contents. You can follow along with the
-`Manual catalog verification <https://docs.lsdb.io/en/stable/tutorials/pre_executed/manual_verification.html>`_.
+`Manual catalog verification <https://docs.lsdb.io/en/stable/tutorials/pre_executed/manual_verification.html>`__.

docs/guide/margin_cache.rst

@@ -1,12 +1,13 @@
 Margin Cache
 ===============================================================================
 
-For more discussion of the whys and hows of margin caches, please see 
-`Max's AAS iPoster <https://aas242-aas.ipostersessions.com/?s=66-E9-54-B6-6B-C3-4B-47-79-24-44-5A-13-25-82-E7>`_
+For more discussion of the whys and hows of margin caches, please see the 
+`demo notebook in LSDB documentation <https://docs.lsdb.io/en/latest/tutorials/margins.html>`__ and
+`Max's AAS iPoster <https://aas242-aas.ipostersessions.com/?s=66-E9-54-B6-6B-C3-4B-47-79-24-44-5A-13-25-82-E7>`__
 for more information.
 
 This page discusses topics around setting up a pipeline to generate a margin
-cache from an existing hats catalog on disk.
+cache from an existing HATS catalog on disk.
 
 At a minimum, you need arguments that include where to find the input files,
 and where to put the output files. A minimal arguments block will look something like:
@@ -91,15 +92,48 @@ This defaults to 5 arcseconds, but you should set this value to whatever is
 appropriate for the astrometry error/PSF width for your instruments. If you're
 not sure how to determine this, please reach out! We'd love to help! :doc:`/guide/contact`.
 
-Setting ``margin_order`` *can* make your pipeline run faster.
-
-#. For each input catalog partition, we can quickly determine all possible 
-   neighboring healpix pixels at the given ``margin_order``. All of these partitions 
-   *may* contain points that are inside the ``margin_threshold``.
-#. For each point in the input catalog, we can quickly determine the healpix
-   pixel at ``margin_order`` and filter points based on this. 
-#. Using this smaller, constrained data set, we do precise boundary checking
-   to determine if the points are within the ``margin_threshold``.
+This is equivalent to setting the ``margin_order``. We use a lookup, with roughly
+the following table of values. This is the minimum separation angle possible within 
+healpix pixels of a given order.
+
+=================  =========================
+``margin_order``   minimum separation angle 
+=================  =========================
+10                 2.15 arcmin
+11                 1.07 arcmin
+12                 32.21 arcsec
+13                 16.10 arcsec
+14                 8.05 arcsec
+15                 4.03 arcsec
+16                 2.01 arcsec
+17                 1.01 arcsec
+18                 0.50 arcsec
+19                 0.25 arcsec
+20                 0.13 arcsec
+21                 62.91 msec
+22                 31.45 msec
+=================  =========================
+
+
+For each input catalog partition, we can quickly determine all possible neighboring
+healpix pixels at the given ``margin_order``. All of these partitions *may* contain 
+points that are inside the ``margin_threshold``. For each point in the input catalog, 
+we can quickly determine the healpix pixel at ``margin_order`` and filter points 
+based on this.
+
+In the figure below, the central yellow pixel is the primary catalog pixel at order 10,
+and the surrounding pink order 13 pixels represent the margin for 10 arcsec.
+
+.. figure:: /static/margin_pixels.png
+   :class: no-scaled-link
+   :align: center
+   :alt: Visual of primary catalog pixel and the margin pixels.
+
+   Visual of primary catalog pixel and the margin pixels.
+
+
+For reasons of runtime performance and numerical precision, we do not perform precise
+boundary checking on individual points.
 
 Progress Reporting
 -------------------------------------------------------------------------------

docs/guide/verification.rst

@@ -125,4 +125,4 @@ Additional Steps
 -------------------------------------------------------------------------------
 
 You can verify and inspect the metadata using methods described in the 
-`Manual catalog verification notebook <https://docs.lsdb.io/en/stable/tutorials/manual_verification.html>`_.
+`Manual catalog verification notebook <https://docs.lsdb.io/en/stable/tutorials/manual_verification.html>`__.

docs/index.rst

@@ -32,7 +32,7 @@ need to install or upgrade versions of dependencies to work with hats-import.
     ``healpy`` is an optional dependency for hats-import (included in the ``full`` extra)
     to support converting from older HiPSCat catalogs, but
     native prebuilt binaries for healpy on Apple Silicon Macs 
-    `do not yet exist <https://healpy.readthedocs.io/en/latest/install.html#binary-installation-with-pip-recommended-for-most-other-python-users>`_, 
+    `do not yet exist <https://healpy.readthedocs.io/en/latest/install.html#binary-installation-with-pip-recommended-for-most-other-python-users>`__, 
     so it's recommended to install via conda before proceeding to hats-import.
 
     .. code-block:: console