Add support for NIST Public Data Reposity DOIs #442
Conversation
|
@santisoler is there anything else needed for this PR to get a review? (apologies if you're not the right ping, but I saw you had the most recent merge activity) |
|
In the meantime. Do you have a link to the API specification of the NIST data repository? In order to makes our lives easier while maintaining this feature, it's important to have a reference of the responses that the API supports, so we can avoid relying on responses that might change in the future. |
|
No worries. Thanks for the status update. The API information is here, though the request response is not particularly thoroughly specified, it appears: https://data.nist.gov/rmm/#/operations/record I'll open a request with the team in charge of this project to see if they can fix that and get a better "example" response shown on that page. (EDIT: see usnistgov/oar-pdr#353 to track this) To see an example response, you can enter a valid ID in the box on the page (such as |
|
Would it be possible to get this reviewed? |
There was a problem hiding this comment.
Hi @jat255! Thanks again for this PR. I think it's looking good, and I think we can merge it shortly. I left a few comments below, please check them out and let me know what do you think.
The other thing I think we should do before merging this PR is adding the NIST Repos to the list of supported DOI services to the documentation. For example, we should include it to the docstring of pooch.DOIDownloader (see https://www.fatiando.org/pooch/dev/api/generated/pooch.DOIDownloader.html#pooch.DOIDownloader).
If you like to include an example that downloads a file from NIST, that would be great! But that won't be needed to merge this PR.
Let me know if you need any help with this. Looking forward to merge it!
|
|
||
| Notes | ||
| ----- | ||
| After Zenodo migrated to InvenioRDM on Oct 2023, their API changed. The | ||
| checksums for each file listed in the API reference is now an md5 sum. | ||
|
|
||
| This method supports both the legacy and the new API. |
There was a problem hiding this comment.
These notes were intended only for the Zenodo downloader, so we can remove them from here.
| Notes | |
| ----- | |
| After Zenodo migrated to InvenioRDM on Oct 2023, their API changed. The | |
| checksums for each file listed in the API reference is now an md5 sum. | |
| This method supports both the legacy and the new API. |
| downloader(to_download, outfile, None) | ||
|
|
||
|
|
||
| def test_populate_registry_nist_pdr(tmp_path): |
There was a problem hiding this comment.
The registry population requires us to ping the server, so let's decorate this test with @pytest.mark.network:
| def test_populate_registry_nist_pdr(tmp_path): | |
| @pytest.mark.network | |
| def test_populate_registry_nist_pdr(tmp_path): |
This PR implements a DOI downloader for the NIST public data repository at https://data.nist.gov
This is my first attempt at developing on pooch at all, so please let me know if there are any changes needed, but I have included tests, and it is working so far as I can tell. Happy to iterate as needed.
Relevant issues/PRs:
Fixes #441