-
Notifications
You must be signed in to change notification settings - Fork 0
Deposit all data to Zenodo #214
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
nuest
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Very good first draft for the PR - thank you!
data/README.md
Outdated
| --- | ||
|
|
||
| ## Dataset Summary | ||
| - **Total articles:** 28 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor formatting issue: please add empty lines around headlines.
data/README.md
Outdated
| | `timeperiod_enddate` | Coverage end dates (ISO format) | | ||
| | `provenance` | Source/method by which the record was imported/added | | ||
|
|
||
| --- |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No need to put lines here, just let the headlines do the structuring. The lines do not communicate any additional meaning.
data/README.md
Outdated
| ## License | ||
|
|
||
| This record includes: | ||
| - **Data files** under **CC0-1.0** (https://creativecommons.org/publicdomain/zero/1.0/) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add empty lines around lists.
data/optimap-main.zip
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please do not commit the archive.
Please asap remove it from the history, don't worry about preserving specific commits. If it's easier, reset to the previous commit and add the changed files manually again...
optimap/settings.py
Outdated
| DATA_DUMP_INTERVAL_HOURS = 6 | ||
| OPENALEX_MAILTO = "[email protected]" | ||
|
|
||
| ZENODO_SANDBOX_TOKEN = os.getenv("M9Ps36SO2dlBJNlMOJMsLWzL9G8b6REY8QSsejUo3Ge6gNXQFRMFe915npTT") # put your sandbox token in env |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please check this file for uses of the env(..) function and do it the same way here, without providing a default.
If there is no token, the code should stop with an error message (using sys.exit("...")).
…_Zenodo_63' into backup/enhancement/Create_new_record_on_Zenodo_63-before-pull
|
Implemented all requested changes except the two called out below. Also moved markdown into requirements.txt, removed the committed ZIP from history, renamed the command to deposit_zenodo, added logging, validated API base (no trailing /), centralized cache path via settings, and split the large test into render/deposit tests. Still pending from the review: TWe currently validate the Zenodo token at command runtime (inside deposit_zenodo) rather than failing in settings.py at import time. We log a concise diff (changed keys + counts); a full pretty-printed JSON diff is not implemented. |
Closes #63
Updates an existing draft only (requires --deposition-id or a setting).
Never publishes (manual step stays in Zenodo UI).
Uploads: README.md, optimap-main.zip, latest GeoJSON (.gz) and GeoPackage (.gpkg) (reuses existing dump tasks/paths).
Merges metadata non-destructively (no clobber):
Reads data/zenodo_dynamic.json and merges: version, related_identifiers (live dump links + “describes” per source).
Does NOT touch stable fields: creators, grants/funders, language, contributors, communities, upload_type, DOI, publisher.
Robust error handling around /actions/edit, 4xx/5xx, and bucket uploads.
How it works -Admin runs:
First command creates/updates:
Second command: