Add generic observation processes which combine the convolution with the noise model. #644
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cdc-mitzimorris
requested review from
SamuelBrand1,
damonbayer and
dylanhmorris
as code owners
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #644 +/- ##
==========================================
+ Coverage 96.98% 97.13% +0.15%
==========================================
Files 42 48 +6
Lines 1094 1326 +232
==========================================
+ Hits 1061 1288 +227
- Misses 33 38 +5
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Thank you for your contribution @pre-commit-ci[bot] 🚀! Your github-pages is ready for download 👉 here 👈! |
…ents, 'aggregate' instead of 'jurisdiction'
…ents, 'aggregate' instead of 'jurisdiction'
for more information, see https://pre-commit.ci
… into mem_generic_observations
for more information, see https://pre-commit.ci
| Count observation processes model the lag between infections and an observed outcome such as hospital admissions, emergency department visits, confirmed cases, or deaths. | ||
| Observed data can be aggregated or available as subpopulation-level counts, which are modeled by classes `Counts` and `CountsBySubpop`, respectively. | ||
|
|
||
| Count observation processes transform infections into expected observed counts by applying an ascertainment rate and convolving with a delay distribution. |
Collaborator
There was a problem hiding this comment.
Suggested change
| Count observation processes transform infections into expected observed counts by applying an ascertainment rate and convolving with a delay distribution. | |
| Count observation processes transform infections into predicted counts by applying an event probability and/or ascertainment rate and convolving with a delay distribution. |
|
|
||
| Count observation processes transform infections into expected observed counts by applying an ascertainment rate and convolving with a delay distribution. | ||
|
|
||
| The expected observations on day $t$ are: |
Collaborator
There was a problem hiding this comment.
Suggested change
| The expected observations on day $t$ are: | |
| The predicted observations on day $t$ are: |
| where: | ||
|
|
||
| - $I_{t-d}$ is the number of incident (new) infections on day $t-d$ (i.e., $d$ days before day $t$) | ||
| - $\alpha$ is the ascertainment rate (e.g., infection-hospitalization ratio) |
Collaborator
There was a problem hiding this comment.
Suggested change
| - $\alpha$ is the ascertainment rate (e.g., infection-hospitalization ratio) | |
| - $\alpha$ is the rate of ascertained counts per infection (e.g., infection-to-hospital admission rate). This can model a mix of biological effects (e.g. some percentage of infections lead to hospital admissions, but not all) and reporting effects (e.g. some percentage of admissions that occur are reported, but not all). |
| - $p_d$ is the delay distribution from infection to observation, conditional on an infection leading to an observation | ||
| - $D$ is the maximum delay | ||
|
|
||
| Discrete observations are generated by sampling from a noise distribution—either Poisson or negative binomial—to model reporting variability. |
Collaborator
There was a problem hiding this comment.
Suggested change
| Discrete observations are generated by sampling from a noise distribution—either Poisson or negative binomial—to model reporting variability. | |
| Discrete observations are generated by sampling from a noise distribution—e.g. Poisson or negative binomial—to model reporting variability. |
| Discrete observations are generated by sampling from a noise distribution—either Poisson or negative binomial—to model reporting variability. | ||
| Poisson assumes variance equals the mean; negative binomial accommodates the overdispersion common in surveillance data. | ||
|
|
||
| **Note on terminology:** In real-world inference, infections are *latent* (unobserved) and must be estimated from observed data like hospital admissions. In this tutorial, we simulate the observation process by specifying infections directly and showing how they produce hospital admissions through convolution and sampling. |
Collaborator
There was a problem hiding this comment.
Suggested change
| **Note on terminology:** In real-world inference, infections are *latent* (unobserved) and must be estimated from observed data like hospital admissions. In this tutorial, we simulate the observation process by specifying infections directly and showing how they produce hospital admissions through convolution and sampling. | |
| **Note on terminology:** In real-world inference, incident infections are typically a *latent* (unobserved) quantity and must be estimated from observed data like hospital admissions. In this tutorial, we simulate the observation process by specifying infections directly and showing how they produce hospital admissions through convolution and sampling. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds work that was done in https://github.com/cdcent/cfa-pyrenew-hierarchical/pull/4 to PyRenew.
It adds the base observation process class, concrete implementations for Count processes and the abstract base class for Measurement processes, together with unit tests and two new tutorials for count and measurement observation processes respectively.
Once this PR and the work done in https://github.com/cdcent/cfa-pyrenew-hierarchical/pull/5 have been added to PyRenew, subsequent PRs will deprecate unused features and harmonize the documentation and tutorials.