Skip to content

Conversation

@nm3224
Copy link
Collaborator

@nm3224 nm3224 commented Oct 20, 2025

changes & context

  • putting in guard rails for gateway courses <=200 to not auto populate config - instead check with school and manually add to config
  • trying to save out logs for all steps (including for inference)
  • refactoring log saving code so it's less redundant
  • created a training and an inference folder under the job/run id for organization
  • adds cohort/term breakdown for checkpoint logic
  • shows what subsets of checkpoint df, selected students df and target df end up in final dataset
  • added missing grade feature to feature table

@nm3224 nm3224 changed the base branch from main to develop October 20, 2025 15:56
@nm3224 nm3224 changed the title Improve data audit Improve data audit + organize job folder Oct 22, 2025
@nm3224
Copy link
Collaborator Author

nm3224 commented Oct 24, 2025

@vishpillai123 this is good to merge now!

Copy link
Collaborator

@vishpillai123 vishpillai123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok we're almost there Noreen! Great work so far on the gateway courses implementation & the logging. I also just looked at one of the schools you ran recently, and the logs look perfect!!! Nice job.

Just wanted to see if the model prep changes were needed now, or if we can wait till later, especially if we haven't tested the changes yet.

@vishpillai123 vishpillai123 merged commit 69e4266 into develop Oct 28, 2025
5 checks passed
@vishpillai123 vishpillai123 deleted the improve_data_audit branch October 28, 2025 14:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants