Skip to content

Conversation

@BharatVe
Copy link
Collaborator

@BharatVe BharatVe commented Aug 26, 2025

Closes #63

Updates an existing draft only (requires --deposition-id or a setting).
Never publishes (manual step stays in Zenodo UI).
Uploads: README.md, optimap-main.zip, latest GeoJSON (.gz) and GeoPackage (.gpkg) (reuses existing dump tasks/paths).
Merges metadata non-destructively (no clobber):
Reads data/zenodo_dynamic.json and merges: version, related_identifiers (live dump links + “describes” per source).
Does NOT touch stable fields: creators, grants/funders, language, contributors, communities, upload_type, DOI, publisher.
Robust error handling around /actions/edit, 4xx/5xx, and bucket uploads.

How it works -Admin runs:

python manage.py render_zenodo_desc
python manage.py deploy_zenodo --confirm --deposition-id <draft_id>

First command creates/updates:

  • data/optimap-main.zip
  • data/README.md
  • data/zenodo_dynamic.json

Second command:

  • Reads those files, merges dynamic metadata, uploads files to the existing sandbox draft.
  • Leaves stable metadata untouched and does not publish.

Copy link
Member

@nuest nuest left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very good first draft for the PR - thank you!

data/README.md Outdated
---

## Dataset Summary
- **Total articles:** 28
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor formatting issue: please add empty lines around headlines.

data/README.md Outdated
| `timeperiod_enddate` | Coverage end dates (ISO format) |
| `provenance` | Source/method by which the record was imported/added |

---
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to put lines here, just let the headlines do the structuring. The lines do not communicate any additional meaning.

data/README.md Outdated
## License

This record includes:
- **Data files** under **CC0-1.0** (https://creativecommons.org/publicdomain/zero/1.0/)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add empty lines around lists.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please do not commit the archive.

Please asap remove it from the history, don't worry about preserving specific commits. If it's easier, reset to the previous commit and add the changed files manually again...

https://www.baeldung.com/ops/git-remove-file-commit-history

DATA_DUMP_INTERVAL_HOURS = 6
OPENALEX_MAILTO = "[email protected]"

ZENODO_SANDBOX_TOKEN = os.getenv("M9Ps36SO2dlBJNlMOJMsLWzL9G8b6REY8QSsejUo3Ge6gNXQFRMFe915npTT") # put your sandbox token in env
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check this file for uses of the env(..) function and do it the same way here, without providing a default.

If there is no token, the code should stop with an error message (using sys.exit("...")).

@BharatVe
Copy link
Collaborator Author

Implemented all requested changes except the two called out below. Also moved markdown into requirements.txt, removed the committed ZIP from history, renamed the command to deposit_zenodo, added logging, validated API base (no trailing /), centralized cache path via settings, and split the large test into render/deposit tests.

Still pending from the review:

TWe currently validate the Zenodo token at command runtime (inside deposit_zenodo) rather than failing in settings.py at import time.

We log a concise diff (changed keys + counts); a full pretty-printed JSON diff is not implemented.

@BharatVe BharatVe marked this pull request as ready for review September 26, 2025 22:55
@nuest nuest changed the title Issue #63 - deploy_zenodo Deposit all data to Zenodo Oct 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create new record on Zenodo with the latest data dumps in different formats

3 participants