Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue/164: Upgrade harmony-py to avoid 413 request entity too large issue #165

Merged
merged 2 commits into from
Mar 14, 2024

Conversation

skorper
Copy link
Contributor

@skorper skorper commented Mar 14, 2024

See #164

The Harmony team released a new version which uses a POST request instead of a GET request when submitting Harmony jobs, which fixes our "413 request entity too large" error.

Testing:

poetry run python3 subscriber/podaac_data_subscriber.py -c SWOT_L2_LR_SSH_BASIC_2.0 -d ./data --start-date 2023-05-01T00:00:00Z --end-date 2024-12-21T00:00:00Z --subset -b="120,-30,160,20"
[2024-03-14 10:41:58,354] {podaac_data_subscriber.py:183} WARNING - No .update__SWOT_L2_LR_SSH_BASIC_2.0 in the data directory. (Is this the first run?)
[2024-03-14 10:42:07,065] {subsetting.py:102} INFO - Waiting for Harmony subsetting job to complete...
./data/SWOT_L2_LR_SSH_Basic_006_547_20231122T012915_20231122T022044_PIC0_01_subsetted.nc4
...
./data/SWOT_L2_LR_SSH_Basic_012_019_20240307T091643_20240307T100711_PIC0_01_subsetted.nc4
[2024-03-14 10:57:00,828] {podaac_data_subscriber.py:298} INFO - END

(removed lines from output for brevity)

@skorper skorper merged commit e5f8205 into release/1.15.1 Mar 14, 2024
skorper added a commit that referenced this pull request Mar 20, 2024
* initial implementaiton of subset code

* added tests, more updates

* removed skip_errors (not implemented yet)

* Upgrade to harmony-py 0.4.9

* subsetting capability initial commit

* update spacing

* Update docs

* update poetry lock

* upgrade to poetry 1.5.1 (latest)

* fix failing checksum test

* fixed regression test issues

* bump poetry version in second gh action file

* Update poetry version to fix build issue

* bump poetry version

* bump poetry version

* bump version to 1.15.0

* bump version to 1.15.0

* add unreleased section to changelog

* issue/161: Fix bug where --subset flag fails with data subscriber (#162)

* fix bug where the subset flag fails for subscriber

* move arg validation to correct spot

* bump version

* bump version

* Issue/164: Upgrade harmony-py to avoid 413 request entity too large issue (#165)

* bump harmony-py version to 0.4.12

* /version 1.15.1-alpha.2

---------

Co-authored-by: mike-gangl <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants