Skip to content

Add PharmaFormer model#336

Merged
nictru merged 12 commits intodaisybio:developmentfrom
Orakl-Oncology-Dev:pharmaformer
Jan 14, 2026
Merged

Add PharmaFormer model#336
nictru merged 12 commits intodaisybio:developmentfrom
Orakl-Oncology-Dev:pharmaformer

Conversation

@nictru
Copy link
Collaborator

@nictru nictru commented Jan 12, 2026

A Transformer-based deep learning model designed to predict clinical drug responses
by integrating gene expression profiles and drug molecular structures.

Original authors: Zhou et al. (2025, 10.1038/s41698-025-01082-6)
Code adapted from their Github: https://github.com/zhouyuru1205/PharmaFormer


I investigated their source code thoroughly (as it was one of the most promising models for my internship task) and found the following problems in their evaluation:

  1. Uses LPO fold creation
  2. Does not account for Simpson's paradox in evaluation

Within drevalpy these parts are handled by the library, allowing us to see the actual performance, which is pretty bad.

@nictru nictru marked this pull request as ready for review January 12, 2026 14:16
@codecov-commenter
Copy link

codecov-commenter commented Jan 12, 2026

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 90.80780% with 33 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (development@8ad96bc). Learn more about missing BASE report.

Files with missing lines Patch % Lines
drevalpy/models/PharmaFormer/pharmaformer.py 94.27% 11 Missing ⚠️
tests/conftest.py 63.33% 11 Missing ⚠️
.../featurizer/create_pharmaformer_drug_embeddings.py 88.57% 8 Missing ⚠️
tests/test_featurizers.py 86.95% 3 Missing ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@              Coverage Diff               @@
##             development     #336   +/-   ##
==============================================
  Coverage               ?   82.23%           
==============================================
  Files                  ?       65           
  Lines                  ?     6795           
  Branches               ?        0           
==============================================
  Hits                   ?     5588           
  Misses                 ?     1207           
  Partials               ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@nictru
Copy link
Collaborator Author

nictru commented Jan 13, 2026

Yes they seem to have made very questionable choices, and I followed them very closely

If you don't evaluate properly it is not necessary to have a meaningful model to achieve 'good' results and end up in Nature

@PascalIversen
Copy link
Collaborator

Ah, ok, wow!!! Well in that case, I am looking forward to seeing how it performs when properly evaluated :D

PascalIversen
PascalIversen previously approved these changes Jan 13, 2026
@nictru
Copy link
Collaborator Author

nictru commented Jan 13, 2026

I don't get a 'Merge' button, so if someone could please click it for me 😄

@JudithBernett
Copy link
Contributor

I don't get a 'Merge' button, so if someone could please click it for me 😄

After you've added it to the docs :D

Copy link
Contributor

@JudithBernett JudithBernett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add to docs please :)

Copy link
Contributor

@JudithBernett JudithBernett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@nictru nictru merged commit 155d732 into daisybio:development Jan 14, 2026
20 checks passed
@nictru nictru deleted the pharmaformer branch January 14, 2026 10:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants