Skip to content

Commit 7a552fe

Browse files
carlynvandykemelange396dshemetov
authored
Add epidatr blog post (#912)
* Create content/blog/2023-09-27-epidatr.Rmd * Add epidatr blog images * rendered html --------- Co-authored-by: george haff <[email protected]> Co-authored-by: Dmitry Shemetov <[email protected]>
1 parent 3e0676a commit 7a552fe

File tree

4 files changed

+247
-0
lines changed

4 files changed

+247
-0
lines changed

‎content/blog/2023-09-27-epidatr.Rmd

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
---
2+
title: New package released! Epidatr, an R client for Delphi Epidata API
3+
author: Dmitry Shemetov, David Weber
4+
date: 2023-09-27
5+
tags:
6+
- r
7+
- epidata
8+
- covidcast
9+
authors:
10+
- dmitry
11+
- davidweb
12+
heroImage: blog-lg-epidatr.jpg
13+
heroImageThumb: blog-thumb-epidatr.jpg
14+
summary: |
15+
`epidatr` is designed to streamline the downloading and usage of data from the [Delphi Epidata API](https://cmu-delphi.github.io/delphi-epidata/). It provides a simple R interface to the API, including functions for downloading data, parsing the results, and converting the data into a tidy format.
16+
17+
output:
18+
blogdown::html_page:
19+
toc: true
20+
---
21+
22+
The [Delphi Epidata API](https://cmu-delphi.github.io/delphi-epidata/) provides
23+
real-time access to epidemiological surveillance data for influenza, COVID-19,
24+
and other diseases from both official government sources such as the [Centers
25+
for Disease Control and Prevention
26+
(CDC)](https://www.cdc.gov/datastatistics/index.html), private partners such as
27+
[Facebook (now
28+
Meta)](https://delphi.cmu.edu/blog/2020/08/26/covid-19-symptom-surveys-through-facebook/)
29+
and [Change Healthcare](https://www.changehealthcare.com/), and other public
30+
datasets like [Google
31+
Trends](https://console.cloud.google.com/marketplace/product/bigquery-public-datasets/covid19-search-trends).
32+
It is built and maintained by the Carnegie Mellon University [Delphi Research
33+
Group](https://delphi.cmu.edu/).
34+
35+
Today we introduce the R package `epidatr`, available [on
36+
CRAN](https://cloud.r-project.org/web/packages/epidatr/index.html), with the
37+
source and development [on github](https://github.com/cmu-delphi/epidatr).
38+
39+
This package is designed to streamline the downloading and usage of data from
40+
the Delphi Epidata API. It provides a simple R interface to the API, including
41+
functions for downloading data, parsing the results, and converting the data
42+
into a tidy format. The API stores a historical record of all data, including
43+
corrections and updates, which is particularly useful for accurately backtesting
44+
forecasting models. We also provide packages for downstream data processing
45+
([epiprocess](https://github.com/cmu-delphi/epiprocess)) and modeling
46+
([epipredict](https://github.com/cmu-delphi/epipredict)).
47+
48+
## Usage
49+
50+
```
51+
library(epidatr)
52+
# Obtain the smoothed covid-like illness (CLI) signal from Delphi's US COVID-19
53+
# Trends and Impact Survey (CTIS), in partnership with Facebook, as it was on
54+
# April 10, 2021 for the US at the national level
55+
epidata <- pub_covidcast(
56+
source = "fb-survey",
57+
signals = "smoothed_cli",
58+
geo_type = "nation",
59+
time_type = "day",
60+
geo_values = "us",
61+
time_values = epirange(20210101, 20210601),
62+
as_of = 20210601
63+
)
64+
epidata
65+
```
66+
67+
```
68+
# A tibble: 151 × 15
69+
geo_value signal source geo_type time_type time_value
70+
<chr> <chr> <chr> <fct> <fct> <date>
71+
1 us smoothed_cli fb-su… nation day 2021-01-01
72+
2 us smoothed_cli fb-su… nation day 2021-01-02
73+
3 us smoothed_cli fb-su… nation day 2021-01-03
74+
4 us smoothed_cli fb-su… nation day 2021-01-04
75+
5 us smoothed_cli fb-su… nation day 2021-01-05
76+
6 us smoothed_cli fb-su… nation day 2021-01-06
77+
7 us smoothed_cli fb-su… nation day 2021-01-07
78+
8 us smoothed_cli fb-su… nation day 2021-01-08
79+
9 us smoothed_cli fb-su… nation day 2021-01-09
80+
10 us smoothed_cli fb-su… nation day 2021-01-10
81+
# ℹ 141 more rows
82+
# ℹ 9 more variables: direction <dbl>, issue <date>,
83+
# lag <dbl>, missing_value <dbl>, missing_stderr <dbl>,
84+
# missing_sample_size <dbl>, value <dbl>, stderr <dbl>,
85+
# sample_size <dbl>
86+
# ℹ Use `print(n = ...)` to see more rows
87+
```
88+
89+
## Installation
90+
91+
Installing the package is straightforward.
92+
93+
```
94+
# Install the CRAN version
95+
pak::pkg_install("epidatr")
96+
97+
# Install the development version from the GitHub dev branch
98+
pak::pkg_install("cmu-delphi/epidatr@dev")
99+
```
100+
101+
### API Keys
102+
103+
The Delphi API requires a (free) API key for full functionality. To generate
104+
your key, register for a pseudo-anonymous account
105+
[here](https://api.delphi.cmu.edu/epidata/admin/registration_form) and see more
106+
discussion on the [general API
107+
website](https://cmu-delphi.github.io/delphi-epidata/api/api_keys.html). The
108+
`epidatr` client will automatically look for this key in the environment
109+
variable `DELPHI_EPIDATA_KEY`. We recommend storing your key in your `.Renviron`
110+
file, which R will read by default.
111+
112+
Note that for the time being, the private endpoints (i.e. those prefixed with
113+
`pvt`) will require a separate key that needs to be passed as an argument.
114+
115+
## For users of the covidcast R package
116+
117+
The `covidcast` package is deprecated and will no longer be updated. The
118+
`epidatr` package is a complete rewrite of the [`covidcast`
119+
package](https://cmu-delphi.github.io/covidcast/covidcastR/), with a focus on
120+
speed, reliability, and ease of use. It also supports more endpoints and data
121+
sources than `covidcast`. When migrating from that package, you will need to use
122+
the
123+
[`pub_covidcast`](https://cmu-delphi.github.io/epidatr/reference/pub_covidcast.html)
124+
function in `epidatr`.

‎content/blog/2023-09-27-epidatr.html

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
---
2+
title: New package released! Epidatr, an R client for Delphi Epidata API
3+
author: Dmitry Shemetov, David Weber
4+
date: 2023-09-27
5+
tags:
6+
- r
7+
- epidata
8+
- covidcast
9+
authors:
10+
- dmitry
11+
- davidweb
12+
heroImage: blog-lg-epidatr.jpg
13+
heroImageThumb: blog-thumb-epidatr.jpg
14+
summary: |
15+
`epidatr` is designed to streamline the downloading and usage of data from the [Delphi Epidata API](https://cmu-delphi.github.io/delphi-epidata/). It provides a simple R interface to the API, including functions for downloading data, parsing the results, and converting the data into a tidy format.
16+
17+
output:
18+
blogdown::html_page:
19+
toc: true
20+
---
21+
22+
<div id="TOC">
23+
<ul>
24+
<li><a href="#usage">Usage</a></li>
25+
<li><a href="#installation">Installation</a><ul>
26+
<li><a href="#api-keys">API Keys</a></li>
27+
</ul></li>
28+
<li><a href="#for-users-of-the-covidcast-r-package">For users of the covidcast R package</a></li>
29+
</ul>
30+
</div>
31+
32+
<p>The <a href="https://cmu-delphi.github.io/delphi-epidata/">Delphi Epidata API</a> provides
33+
real-time access to epidemiological surveillance data for influenza, COVID-19,
34+
and other diseases from both official government sources such as the <a href="https://www.cdc.gov/datastatistics/index.html">Centers
35+
for Disease Control and Prevention
36+
(CDC)</a>, private partners such as
37+
<a href="https://delphi.cmu.edu/blog/2020/08/26/covid-19-symptom-surveys-through-facebook/">Facebook (now
38+
Meta)</a>
39+
and <a href="https://www.changehealthcare.com/">Change Healthcare</a>, and other public
40+
datasets like <a href="https://console.cloud.google.com/marketplace/product/bigquery-public-datasets/covid19-search-trends">Google
41+
Trends</a>.
42+
It is built and maintained by the Carnegie Mellon University <a href="https://delphi.cmu.edu/">Delphi Research
43+
Group</a>.</p>
44+
<p>Today we introduce the R package <code>epidatr</code>, available <a href="https://cloud.r-project.org/web/packages/epidatr/index.html">on
45+
CRAN</a>, with the
46+
source and development <a href="https://github.com/cmu-delphi/epidatr">on github</a>.</p>
47+
<p>This package is designed to streamline the downloading and usage of data from
48+
the Delphi Epidata API. It provides a simple R interface to the API, including
49+
functions for downloading data, parsing the results, and converting the data
50+
into a tidy format. The API stores a historical record of all data, including
51+
corrections and updates, which is particularly useful for accurately backtesting
52+
forecasting models. We also provide packages for downstream data processing
53+
(<a href="https://github.com/cmu-delphi/epiprocess">epiprocess</a>) and modeling
54+
(<a href="https://github.com/cmu-delphi/epipredict">epipredict</a>).</p>
55+
<div id="usage" class="section level2">
56+
<h2>Usage</h2>
57+
<pre><code>library(epidatr)
58+
# Obtain the smoothed covid-like illness (CLI) signal from Delphi&#39;s US COVID-19
59+
# Trends and Impact Survey (CTIS), in partnership with Facebook, as it was on
60+
# April 10, 2021 for the US at the national level
61+
epidata &lt;- pub_covidcast(
62+
source = &quot;fb-survey&quot;,
63+
signals = &quot;smoothed_cli&quot;,
64+
geo_type = &quot;nation&quot;,
65+
time_type = &quot;day&quot;,
66+
geo_values = &quot;us&quot;,
67+
time_values = epirange(20210101, 20210601),
68+
as_of = 20210601
69+
)
70+
epidata</code></pre>
71+
<pre><code># A tibble: 151 × 15
72+
geo_value signal source geo_type time_type time_value
73+
&lt;chr&gt; &lt;chr&gt; &lt;chr&gt; &lt;fct&gt; &lt;fct&gt; &lt;date&gt;
74+
1 us smoothed_cli fb-su… nation day 2021-01-01
75+
2 us smoothed_cli fb-su… nation day 2021-01-02
76+
3 us smoothed_cli fb-su… nation day 2021-01-03
77+
4 us smoothed_cli fb-su… nation day 2021-01-04
78+
5 us smoothed_cli fb-su… nation day 2021-01-05
79+
6 us smoothed_cli fb-su… nation day 2021-01-06
80+
7 us smoothed_cli fb-su… nation day 2021-01-07
81+
8 us smoothed_cli fb-su… nation day 2021-01-08
82+
9 us smoothed_cli fb-su… nation day 2021-01-09
83+
10 us smoothed_cli fb-su… nation day 2021-01-10
84+
# ℹ 141 more rows
85+
# ℹ 9 more variables: direction &lt;dbl&gt;, issue &lt;date&gt;,
86+
# lag &lt;dbl&gt;, missing_value &lt;dbl&gt;, missing_stderr &lt;dbl&gt;,
87+
# missing_sample_size &lt;dbl&gt;, value &lt;dbl&gt;, stderr &lt;dbl&gt;,
88+
# sample_size &lt;dbl&gt;
89+
# ℹ Use `print(n = ...)` to see more rows</code></pre>
90+
</div>
91+
<div id="installation" class="section level2">
92+
<h2>Installation</h2>
93+
<p>Installing the package is straightforward.</p>
94+
<pre><code># Install the CRAN version
95+
pak::pkg_install(&quot;epidatr&quot;)
96+
97+
# Install the development version from the GitHub dev branch
98+
pak::pkg_install(&quot;cmu-delphi/epidatr@dev&quot;)</code></pre>
99+
<div id="api-keys" class="section level3">
100+
<h3>API Keys</h3>
101+
<p>The Delphi API requires a (free) API key for full functionality. To generate
102+
your key, register for a pseudo-anonymous account
103+
<a href="https://api.delphi.cmu.edu/epidata/admin/registration_form">here</a> and see more
104+
discussion on the <a href="https://cmu-delphi.github.io/delphi-epidata/api/api_keys.html">general API
105+
website</a>. The
106+
<code>epidatr</code> client will automatically look for this key in the environment
107+
variable <code>DELPHI_EPIDATA_KEY</code>. We recommend storing your key in your <code>.Renviron</code>
108+
file, which R will read by default.</p>
109+
<p>Note that for the time being, the private endpoints (i.e. those prefixed with
110+
<code>pvt</code>) will require a separate key that needs to be passed as an argument.</p>
111+
</div>
112+
</div>
113+
<div id="for-users-of-the-covidcast-r-package" class="section level2">
114+
<h2>For users of the covidcast R package</h2>
115+
<p>The <code>covidcast</code> package is deprecated and will no longer be updated. The
116+
<code>epidatr</code> package is a complete rewrite of the <a href="https://cmu-delphi.github.io/covidcast/covidcastR/"><code>covidcast</code>
117+
package</a>, with a focus on
118+
speed, reliability, and ease of use. It also supports more endpoints and data
119+
sources than <code>covidcast</code>. When migrating from that package, you will need to use
120+
the
121+
<a href="https://cmu-delphi.github.io/epidatr/reference/pub_covidcast.html"><code>pub_covidcast</code></a>
122+
function in <code>epidatr</code>.</p>
123+
</div>

‎content/blog/images/blog-lg-epidatr.jpg

224 KB
Loading

‎content/blog/images/blog-thumb-epidatr.jpg

48.2 KB
Loading

0 commit comments

Comments
 (0)