Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Developer guide for further development. #147

Merged
merged 16 commits into from
Jan 20, 2025
Merged
14 changes: 14 additions & 0 deletions docs/dev_guide/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
.. _sunpy-soar-dev-guide-index:
nabobalis marked this conversation as resolved.
Show resolved Hide resolved

***************
Developer Guide
***************

This guide will walk you through about contributing towards sunpy-soar.
NucleonGodX marked this conversation as resolved.
Show resolved Hide resolved

.. toctree::
:maxdepth: 1

working
tables
query
50 changes: 50 additions & 0 deletions docs/dev_guide/query.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
.. _sunpy-soar-dev-guide-query:

***************************************
Request methods ``sunpy-soar`` supports
***************************************

sunpy-soar currently supports two REQUEST methods: ``doQuery`` and ``doQueryFilteredByDistance``.

``doQuery``: This is the standard method used when no specific distance attribute is included in the search query.
It performs a generalquery based on the provided parameters, retrieving the data that matches the criteria specified.
NucleonGodX marked this conversation as resolved.
Show resolved Hide resolved

``doQueryFilteredByDistance``: This method is employed when a distance parameter is included in the search query.
Unlike doQuery, this method filters the entire query based on the specified distance value.
The time attribute is not required when using doQueryFilteredByDistance,as the time is internally
calculated based on the provided distance. The distance value is appended to the end of the query using &DISTANCE(dmin, dmax),
where dmin and dmax are Astropy units of distance. These values must fall within the range of 0.28 AU to 1.0 AU;
otherwise, the query will not return any results.
NucleonGodX marked this conversation as resolved.
Show resolved Hide resolved

Example:
NucleonGodX marked this conversation as resolved.
Show resolved Hide resolved

.. code-block:: python

import astropy.units as u
import sunpy.net.attrs as a
from sunpy.net import Fido

instrument = a.Instrument("RPW")
level = a.Level(2)
distance = a.soar.Distance(0.28 * u.AU, 0.30 * u.AU)

result = Fido.search(instrument & level & distance)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should check this result is output.


Here the query's REQUEST type to "doQueryFilteredByDistance", which is a special method that filters the entire query
based on the specified distance value. This method is used when a distance parameter is included in the search query,
and the time attribute is not required.
NucleonGodX marked this conversation as resolved.
Show resolved Hide resolved

Example Query:
NucleonGodX marked this conversation as resolved.
Show resolved Hide resolved

.. code-block:: python

Query="SELECT+*+FROM+v_sc_data_item+WHERE+instrument='RPW'+AND+level='L2'&DISTANCE(0.28,0.30)"
NucleonGodX marked this conversation as resolved.
Show resolved Hide resolved

How can other Request methods be added?
=======================================
NucleonGodX marked this conversation as resolved.
Show resolved Hide resolved

To add support for additional request methods, you'll need to consider the impact they will have on the query structure.
The REQUEST parameter in the query must be updated accordingly, and any necessary modifications to the query string should be
made to accommodate the new method. If the new request method requires a specific attribute, you may need to add it as a class
in the attrs.py file (if it doesn't already exist in `sunpy.net.attrs`). Additionally, a walker for this attribute
will need to be created.
NucleonGodX marked this conversation as resolved.
Show resolved Hide resolved
49 changes: 49 additions & 0 deletions docs/dev_guide/tables.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
.. _sunpy-soar-dev-guide-tables:

**********************************
Tables supported in ``sunpy-soar``
**********************************

The ``sunpy-soar`` library currently supports data retrieval from both science and low-latency data tables, such as
``v_sc_data_item`` and ``v_ll_data_item``. Additionally, it provides support for join tables associated with remote
instruments, such as ``v_eui_sc_fits``.
NucleonGodX marked this conversation as resolved.
Show resolved Hide resolved

In the context of ``sunpy-soar``, data tables contain columns related to scientific measurements, while instrument
tables contain metadata specific to particular instruments. The ``sunpy-soar`` library specifically supports the
wavelength and detector columns within these tables. These columns are linked to the data columns in the instrument
tables through a join operation, using the ``data_item_oid`` as the key.
NucleonGodX marked this conversation as resolved.
Show resolved Hide resolved

How can a new column be added?
==============================

If the new column you wish to add is part of an existing data or instrument FITS table, you can extend the
corresponding ADQL query to include this column. For example, if the new column is in the instrument table, it should
be included in the SELECT clause of the query for that table.
NucleonGodX marked this conversation as resolved.
Show resolved Hide resolved

Moreover, if filtering by this new column is required, you must consider adding it as an attribute in the ``attrs.py``
file. If the column already exists within `sunpy.net.attrs`, you should add a corresponding walker in the
``attrs.py`` file. The ``_can_handle_query`` method in the ``client.py`` file should also be updated to ensure proper
support for this new column in queries.
NucleonGodX marked this conversation as resolved.
Show resolved Hide resolved

How can a new table be added?
=============================

When adding support for a new table in ``sunpy-soar``, it is essential to fully understand the context and requirements.
This includes identifying the key that will be used for join operations and determining what data should be
returned to the user.
NucleonGodX marked this conversation as resolved.
Show resolved Hide resolved

For direct joins, the key relationship between the tables should be carefully identified. Once the joins are
established, the necessary columns can be included in the query, following the process outlined for adding columns.
NucleonGodX marked this conversation as resolved.
Show resolved Hide resolved

Depending on the use case, you might need to determine the type of join (e.g., inner join, outer join, left join) and what
specific tables or columns should be displayed to the user. In some cases, conditional logic may be required, where
certain attributes in the search query trigger specific join operations or column selections.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Depending on the use case, you might need to determine the type of join (e.g., inner join, outer join, left join) and what
specific tables or columns should be displayed to the user. In some cases, conditional logic may be required, where
certain attributes in the search query trigger specific join operations or column selections.
Depending on the use case, you might need to determine the type of join (e.g., inner join, outer join, left join) and what specific tables or columns should be displayed to the user.
In some cases, conditional logic may be required, where certain attributes in the search query trigger specific join operations or column selections.

Any examples of the conditional logic?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay, I'll add.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The explanation I provided was specifically in the context of handling joins with the FoV (Field of View) table, where certain columns are returned when the user includes FoV in their search query. However, since the implementation details of this feature are still under discussion, it might be best to omit this section for now.

Once the FoV pull request (PR) is finalized and the implementation becomes clear, I am eager to update the guide with the new capabilities and features that will be introduced.


Finally, ensure that the ``_do_search`` method is updated to reflect these changes. This method should handle any
additional columns or conditional logic for attribute-specific queries, ensuring that the appropriate data is returned
to the user.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Finally, ensure that the ``_do_search`` method is updated to reflect these changes. This method should handle any
additional columns or conditional logic for attribute-specific queries, ensuring that the appropriate data is returned
to the user.
Finally, ensure that the ``_do_search`` method is updated to reflect these changes.
This method should handle any additional columns or conditional logic for attribute-specific queries, ensuring that the appropriate data is returned to the user.

Can we not link to the do_search method?


Note
====

Accordingly, test cases for every extreme and usual case must be added, to show full functionality of the added table or column.
NucleonGodX marked this conversation as resolved.
Show resolved Hide resolved
47 changes: 47 additions & 0 deletions docs/dev_guide/working.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
.. _sunpy-soar-dev-guide-working:

****************
Newcomer's Guide
****************

``sunpy-soar`` is a Python library developed to interact with the SOAR (Solar Orbiter Archive) data
hosted by ESA (European Space Agency). It provides tools and utilities to access, search, download and
analyze data collected by the Solar Orbiter mission.
NucleonGodX marked this conversation as resolved.
Show resolved Hide resolved

How data is retrieved from SOAR
===============================
NucleonGodX marked this conversation as resolved.
Show resolved Hide resolved

To retrieve data from SOAR, you first use the ``Fido`` object from SunPy
and specify the desired attributes using `sunpy.net.attrs`. These
attributes define the criteria for the data you want to retrieve, such as the
time range, instrument, or wavelength. Here is an example of how to specify
the time range for the data you want to retrieve:
NucleonGodX marked this conversation as resolved.
Show resolved Hide resolved

.. code-block:: python

import sunpy.net.attrs as a
from sunpy.net import Fido
import sunpy_soar

instrument = a.Instrument("EUI")
time = a.Time("2021-02-01", "2021-02-02")
level = a.Level(1)
product = a.soar.Product("EUI-FSI174-IMAGE")

result = Fido.search(instrument & time & level & product)


``sunpy-soar`` constructs the query based on the specified criteria, then generates a URL to interact with the SOAR API.
This is done by using the Table Access Protocol (TAP), a widely adopted standard in the astronomical community for accessing large datasets.
nabobalis marked this conversation as resolved.
Show resolved Hide resolved
The results are returned in the form of an Astropy table, providing a structured and efficient format for further analysis and visualization within the Python environment.

A generated query looks like:

.. code-block:: python

without_join="SELECT+*+FROM+v_sc_data_item+WHERE+instrument='EPD'+AND+begin_time>='2021-02-01+00:00:00'+AND+begin_time<='2021-02-02+00:00:00'+AND+level='L1'+AND+descriptor='epd-epthet2-nom-close'"
with_join="SELECT+h1.instrument, h1.descriptor, h1.level, h1.begin_time, h1.end_time, h1.data_item_id, h1.filesize, h1.filename, h1.soop_name, h2.detector, h2.wavelength, h2.dimension_index+"
"FROM+v_sc_data_item AS h1 JOIN v_eui_sc_fits AS h2 USING (data_item_oid)+WHERE+h1.instrument='EUI'+AND+h1.begin_time>='2021-02-01+00:00:00'+AND+h1.begin_time<='2021-02-02+00:00:00'+AND+""
"h2.dimension_index='1'+AND+h1.level='L1'+AND+h1.descriptor='eui-fsi174-image'"
NucleonGodX marked this conversation as resolved.
Show resolved Hide resolved

The url is generated with the query formed based on the parameters, Then Fido is used to print or download the data.
NucleonGodX marked this conversation as resolved.
Show resolved Hide resolved
1 change: 1 addition & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ This will walk you through getting set up for contributing.
:hidden:

generated/gallery/index
dev_guide/index
how_to/index
api
changelog
NucleonGodX marked this conversation as resolved.
Show resolved Hide resolved
Expand Down