Skip to content

Add tolid broker#31

Merged
emilylm merged 5 commits into
mainfrom
add-tolid-broker
Jun 18, 2026
Merged

Add tolid broker#31
emilylm merged 5 commits into
mainfrom
add-tolid-broker

Conversation

@emilylm

@emilylm emilylm commented Jun 18, 2026

Copy link
Copy Markdown
Collaborator

📌 Summary

This PR adds lightweght durable ToLID support for broker workflows in Canopy, it simplifies the initial ToLID request path to work directly from specimen/sample accession lookup, fixes a submitted_at persistence bug for project submissions, updates specimen bulk import to accept the correct nested input shape, and also keeps project metadata aligned when taxonomy info enriches an organism's scientific name.


🔄 Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to change)
  • 🧹 Refactor (code change that neither fixes a bug nor adds a feature)
  • 📚 Documentation update
  • ⚙️ CI / build / tooling change

🧩 Key Changes

  • Added durable tolid_request persistence, broker-facing ToLID endpoints, tests, and broker flow documentation.
  • Simplified the ToLID API so first-time requests no longer depend on pre-populated not_requested rows; broker now looks up specimen metadata directly by accession and Canopy creates ToLID rows lazily on report.
  • Fixed ProjectSubmission.submitted_at ORM mapping so broker updates persist correctly.
  • Updated bulk specimen sample import to accept nested { "<taxon_id>": { "<specimen_id>": {...} } } payloads instead of <taxon_id>_<specimen_id> keys.
  • Added project metadata resync when taxonomy info updates an organism’s effective scientific name, including refresh of draft/ready project submission payloads.
  • Fixed the ToLID migration enum creation so Alembic does not attempt to create tolid_request_status twice.

✅ Checklist

  • Tests are passing locally / in CI
  • New tests have been added for new or changed functionality
  • Self-review completed
  • Documentation updated (if needed)
  • No sensitive data or secrets included

🔍 Review Notes

  • First-time ToLID requests now come from accession lookup, while pending retries still come from persisted tolid_request rows
  • Project creation still happens at organism creation time; taxonomy-info changes now resync project labels instead of deferring project creation, because existing sample flows depend on the genomic_data project already existing.
  • Accepted project submission payloads are left unchanged during taxonomy-driven resync; draft/ready payloads are refreshed.
  • There is currently one organism (tax_id 143878) that does not have organism data outputted from the data mapper, as such it doesn't fetch taxonomy info from ncbi. Need to investigate why this taxon_id is omitted from taxon data. Issue added

📎 Related Issues / Tickets

  • Closes #
  • Related to #

@emilylm
emilylm marked this pull request as ready for review June 18, 2026 11:53
@emilylm
emilylm merged commit cfd4797 into main Jun 18, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant