Skip to content

Conversation

@nmdefries
Copy link
Collaborator

@nmdefries nmdefries commented Feb 18, 2022

Create evaluate_chu-like function for pulling truth data and scoring state predictions. Because the function bypasses evalcast's built-in data-fetching, this should be a lot faster.

Haven't worked on this in a long time, though, so the code will need to be reviewed, and checked for correctness and speed compared to the current pipeline. This may be entirely ready to go as-is 🤞.

@nmdefries
Copy link
Collaborator Author

IIRC (it was a long time ago, though) all state truth data using this approach and using evalcast matched except for California.

@nmdefries
Copy link
Collaborator Author

nmdefries commented Apr 13, 2022

Test run in production failed. Update: Below changes should address this.

@nmdefries
Copy link
Collaborator Author

After adding some fixes, a production run succeeded, with a run time of 4h 25m. This is a reduction of about 1.5h, in line with previous estimates of how long downloading truth data in evalcast takes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants