Release/1.15.0 (#154)

* initial implementaiton of subset code

* added tests, more updates

* removed skip_errors (not implemented yet)

* Upgrade to harmony-py 0.4.9

* subsetting capability initial commit

* update spacing

* Update docs

* update poetry lock

* upgrade to poetry 1.5.1 (latest)

* fix failing checksum test

* fixed regression test issues

* bump poetry version in second gh action file

* Update poetry version to fix build issue

* bump poetry version

* bump poetry version

* bump version to 1.15.0

* bump version to 1.15.0

* Fixed bug introduced in issues/36 where valid bboxes were raising an exception

* bump version to 1.15.0-a.2

* Add error if both cycle and subset are provided

* If verbose, output Harmony job details. Also allow temporal and spatial bounds to be OPTIONAL so users don't have to provide these values.

* Added log message to indicate harmony job is submitted and waiting

* fixed cycle check typo

* dry run + subset

* idl check

* bump version to a3

* fix failing unit test to correct value returned by CMR (#155)

---------

Co-authored-by: mike-gangl <[email protected]>

.github/workflows/python-app.yml

@@ -16,7 +16,7 @@ jobs:
       max-parallel: 2
       matrix:
         python-version: [ "3.7", "3.8", "3.9", "3.10" ]
-        poetry-version: [ "1.1.14" ]
+        poetry-version: [ "1.5.1" ]
         os: [ ubuntu-22.04, macos-latest, windows-latest ]
     runs-on: ${{ matrix.os }}
     steps:
@@ -63,7 +63,7 @@ jobs:
       max-parallel: 1
       matrix:
         python-version: [ "3.7", "3.8", "3.9", "3.10" ]
-        poetry-version: [ "1.1.14" ]
+        poetry-version: [ "1.5.1" ]
         os: [ ubuntu-22.04, macos-latest, windows-latest ]
     runs-on: ${{ matrix.os }}
     steps:

.github/workflows/release.yml

@@ -20,7 +20,7 @@ jobs:
     - name: Install Poetry
       uses: abatilo/[email protected]
       with:
-        poetry-version: 1.1.14
+        poetry-version: '1.5.1'
     - name: Install dependencies
       run: |
         poetry install

CHANGELOG.md

@@ -3,6 +3,10 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 
+## [1.15.0]
+### Added
+- Added support for Harmony subsetting
+
 ## [1.14.0]
 ### Added
 - Added support for wildcard search patterns in podaac-data-downloader when executed with the -gr option (i.e. search/download by CMR Granule Ur/Id). Also, added usage details to Downloader.md to describe this new feature [138](https://github.com/podaac/data-subscriber/pull/138).

Downloader.md

@@ -41,6 +41,7 @@ optional arguments:
                         Specify a provider for collection search. Default is POCLOUD.
   --limit LIMIT         Integer limit for number of granules to download. Useful in testing. Defaults to no limit.
   --dry-run             Search and identify files to download, but do not actually download them
+  --subset              Flag to enable subsetting on the specified collection 
 
 ```
 
@@ -236,7 +237,9 @@ and
 
 Using the `--process` option, you can run a simple command agaisnt the "just" downloaded file. This will take the format of "<command> <path/to/file>". This means you can run a command like `--process gzip` to gzip all downloaded files. We do not support more advanced processes at this time (piping, running a process on a directory, etc).
 
+### Granule subsetting
 
+To enable granule subsetting, include the `--subset` flag in your request. This will invoke the NASA Harmony API to subset the granules in the specified collection. The collection must have subsetting enabled for this feature to function. If it does not, the data will be downloaded normally. 
 
 ### In need of Help?
 The PO.DAAC User Services Office is the primary point of contact for answering your questions concerning data and information held by the PO.DAAC. User Services staff members are knowledgeable about both the data ordering system and the data products themselves. We answer questions about data, route requests to other DAACs, and direct questions we cannot answer to the appropriate information source.

Subscriber.md

@@ -35,6 +35,7 @@ $> podaac-data-subscriber -h
 -p PROVIDER, --provider PROVIDER
                       Specify a provider for collection search. Default is POCLOUD.
 --dry-run             Search and identify files to download, but do not actually download them
+--subset              Flag to enable subsetting on the specified collection
 ```
 
 ## Run the Script
@@ -212,6 +213,9 @@ and
 
 Using the `--process` option, you can run a simple command agaisnt the "just" downloaded file. This will take the format of "<command> <path/to/file>". This means you can run a command like `--process gzip` to gzip all downloaded files. We do not support more advanced processes at this time (piping, running a process on a directory, etc).
 
+### Granule subsetting
+
+To enable granule subsetting, include the `--subset` flag in your request. This will invoke the NASA Harmony API to subset the granules in the specified collection. The collection must have subsetting enabled for this feature to function. If it does not, the data will be downloaded normally.
 
 ### In need of Help?
 The PO.DAAC User Services Office is the primary point of contact for answering your questions concerning data and information held by the PO.DAAC. User Services staff members are knowledgeable about both the data ordering system and the data products themselves. We answer questions about data, route requests to other DAACs, and direct questions we cannot answer to the appropriate information source.

pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "podaac-data-subscriber"
-version = "1.14.0"
+version = "1.15.0-alpha.3"
 description = "PO.DAAC Data Subscriber Command Line Tool"
 authors = ["PO.DAAC <[email protected]>"]
 readme = "README.md"
@@ -16,6 +16,7 @@ python = "^3.7"
 requests = "^2.27.1"
 tenacity = "^8.0.1"
 packaging = "^23.0"
+harmony-py = "^0.4.9"
 
 [tool.poetry.dev-dependencies]
 pytest = "^7.1.2"