Skip to content

feat: add ncbi taxonomy information to database#23

Merged
emilylm merged 15 commits into
mainfrom
add-ncbi-taxon-info
May 18, 2026
Merged

feat: add ncbi taxonomy information to database#23
emilylm merged 15 commits into
mainfrom
add-ncbi-taxon-info

Conversation

@emilylm

@emilylm emilylm commented May 18, 2026

Copy link
Copy Markdown
Collaborator

📌 Summary

This PR adds NCBI taxonomy enrichment support to the API and updates the organism/taxonomy data model to separate BPA-supplied organism fields from NCBI-derived taxonomy fields.

It introduces the NCBI lookup service, stores fetched taxonomy details on taxonomy_info, updates the relevant models and schemas, and adds bulk-import support plus validation and tests around the new behavior.

We also re-set 0001-initial-schema.sql and clear previous db migrations


🔄 Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to change)
  • 🧹 Refactor (code change that neither fixes a bug nor adds a feature)
  • ⚙️ CI / build / tooling change

🧩 Key Changes

  • Added an Alembic migration and schema/model updates to move NCBI taxonomy detail fields onto taxonomy_info and rename organism source fields to bpa_*.
  • Added ncbi_taxonomy_service to fetch and map taxonomy details from the NCBI Datasets API, including lineage enrichment and mito_ref. This code is largely across from https://github.com/emilylm/tutorial-fetch-ncbi-data/blob/main/src/parent_lineage_enrichment.py
  • Updated taxonomy_info create and bulk-import flows to fetch and persist NCBI taxonomy details, with bulk import using batched lookups.
  • Added stricter request validation so ncbi_* fields are not accepted from callers, including a typed bulk taxonomy-info import schema.
  • Updated organism and assembly-related code to align with the new field names and taxonomy-info relationship usage.
  • Added service and endpoint tests covering NCBI enrichment, bulk import behavior, and request validation.
  • Removed previous database migrations and created an immutable 0001-initial-schema.sql

✅ Checklist

  • Tests are passing locally / in CI
  • New tests have been added for new or changed functionality
  • Self-review completed
  • Documentation updated (if needed)
  • No sensitive data or secrets included

🔍 Review Notes

  • This PR includes a schema/API shape change for organism fields: BPA/source values now use bpa_* names.
  • For both the create and bulk-create endpoints, if a row already exists in taxonomy_info with the provided taxon_id, we update the existing taxonomy_info object.

📎 Related Issues / Tickets

  • Closes # AP-396
  • Related to #

@emilylm
emilylm marked this pull request as ready for review May 18, 2026 12:56
@emilylm
emilylm merged commit a4c621f into main May 18, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant