Skip to content

Commit

Permalink
docs: updated figures
Browse files Browse the repository at this point in the history
  • Loading branch information
kirangadhave committed May 29, 2024
1 parent 613c9c3 commit 2205934
Show file tree
Hide file tree
Showing 13 changed files with 114 additions and 50 deletions.
103 changes: 84 additions & 19 deletions docs-website/docs/getting-started/simple-tutorial.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,37 @@
---
sidebar_position: 2
---

import ReactPlayer from 'react-player'
import ContextSensitiveImage from '../../src/components/ContextSensitiveImage.tsx'

# Quickstart Tutorial


## Analysis with Persist

It’s easiest to see how it works by following an analysis. We’ll look at [avalanches in the Utah mountains](https://utahavalanchecenter.org/). You can follow along using this [binder instance.](https://mybinder.org/v2/gh/visdesignlab/persist/HEAD?labpath=examples%2Fblog.ipynb) _Binder instance might take a few minutes to start the first time._ You can also [download the notebook](https://raw.githubusercontent.com/visdesignlab/persist/main/examples/blog.ipynb) and run the notebook in a local JupyterLab instance. Follow the instructions [here](https://vdl.sci.utah.edu/persist/docs/installation) to set up local JupyterLab with Persist extension.

_The notebook uses VegaAltair to create interactive visualizations and assumes some familiarity with VegaAltair, VegaLite, and the declarative approach to creating visualizations. You can refer to their [getting started](https://altair-viz.github.io/getting_started/overview.html) for a quick introduction. You won’t have to write any VegaAltair code to follow the blog._

<ContextSensitiveImage src='img/tutorial_images/step_2.png' className='docs-image'/>
We will load the data in a Pandas dataframe.

```py
import pandas as pd
import altair as alt
import persist_ext as PR # Load the extension

av_ut = pd.read_csv("avalanches_ut.csv") # load the csv
av_ut.head()
```

<ContextSensitiveImage src='img/tutorial_images/step_2.png' className='docs-image'/>

After loading the data, we examine the data in an interactive data table using the following code.

```py
PR.PersistTable(av_ut, df_name="av_ut1")
```

<ContextSensitiveImage src='img/tutorial_images/step_3.png' className='docs-image'/>

### Working with Columns
Expand All @@ -31,70 +45,121 @@ Next, we will delete the coordinates and comments columns since we will not perf

<ContextSensitiveImage src='img/tutorial_images/step_5.png' className='docs-image'/>


### Changing a Column’s Data Type

We can hover over the column headers to see the data type. The `Depth_inches` column has the data type `string` instead of `float`. We want the `Depth_inches` to be a `float` column so we can plot it later. We also see that row 7 has a trailing inches symbol, ``, which is the cause of the incorrect data type.

We can use the search box on the top left of the table to find all instances of the trailing symbol. We can double-click the cell to edit it and remove the symbol. Using the menu in the column header, we will change the column's data type to float.

### Extracting a Dataframe

We can click the “insert dataframe” button in the dataframe manager at the bottom of the table to insert a cell with a pandas dataframe called `av_ut1`. This dataframe has the changes we made in the table applied: the column names are corrected, two columns are removed, and the datatype of `Depth_inches` is numerical.

<ContextSensitiveImage src='img/tutorial_images/step_6.png' className='docs-image'/>

#### Equivalent pandas code
For reference, here is the equivalent pandas code for making these changes to the dataframe:

<ContextSensitiveImage src='img/tutorial_images/step_7.png' className='docs-image'/>
For reference, here is the equivalent pandas code for making these changes to the dataframe:

```py
# Remove leading `;` from column names
av_ut1 = av_ut.rename(
columns={col: col.replace(";", "") for col in av_ut.columns}
)

# Drop two columns
av_ut1 = av_ut1.drop(columns=["Coordinates", "Comments 1"])

# Replace trailing `"` from Depth_inches
av_ut1["Depth_inches"] = av_ut1["Depth_inches"].apply(lambda x: x.replace('"', ''))
# Cast Depth_inches to float
av_ut1["Depth_inches"] = av_ut1["Depth_inches"].astype(float)

av_ut1.head()
```

### Filtering Data in a Visualization
Next, we take a look at how to interactively manipulate data in visualizations.

Using the following code, we will create an interactive scatterplot of `Elevation_feet` vs. `Depth_inches` using the plot module (basically a shorthand for common vega-altair plots) and our new dataframe.
Next, we take a look at how to interactively manipulate data in visualizations.

Using the following code, we will create an interactive scatterplot of `Elevation_feet` vs. `Depth_inches` using the plot module (basically a shorthand for common vega-altair plots) and our new dataframe.

```py
PR.plot.scatterplot(
av_ut1,
x="Elevation_feet:Q",
y="Depth_inches:Q",
df_name="av_ut2"
)
```

<ContextSensitiveImage src='img/tutorial_images/step_8.png' className='docs-image'/>

If we look at this plot carefully, we can see that it shows avalanches occurring at elevations outside the possible range for Utah (Utah’s lowest point is at about 2,200 feet; its highest is at 13,528 feet), indicating that these entries are unreliable. We can select these points using a brush and remove them from the dataset.

We can again access the resulting dataframe from the dataframe manager.

<ContextSensitiveImage src='img/tutorial_images/step_9.png' className='docs-image'/>

#### Equivalent pandas code

Again, here’s the equivalent pandas code:

<ContextSensitiveImage src='img/tutorial_images/step_9.png' className='docs-image'/>
```py
av_ut2 = av_ut1[av_ut1["Elevation_feet"].between(4000, 15000)]
```

### Removing old data
We will not look at the count of records aggregated by the year. We will use the following code to create the plot.
### Creating a New Category in a Custom Vega-Altair Chart

We see that before 2010, we don’t have many records. We will remove the records for those years from our analysis. We can again interactively select the data we want to remove and filter it out.
Next, we’ll add a new categorical classification to our dataset: types of avalanche activity vary over the snow season, so we classify the season into three phases: Early, Peak, Spring. Using the following code, we will create a Vega-Altair bar chart with data aggregated by month and make it persistent:

Equivalent pandas code
```py
# Create an interval selection param
selection = alt.selection_interval(name="selection", encodings=["x"])

### Creating a New Category in a Custom Vega-Altair Chart
Next, we'll add a new categorical classification to our dataset: types of avalanche activity vary over the snow season, so we classify the season into three phases: Start, Middle, End. Using the following code, we will create a Vega-Altair bar chart with data aggregated by month and make it persistent:
chart = alt.Chart(av_ut2).mark_bar().encode( # Create a barchart for `av_ut2`
x=alt.X("Month:O").sort([10]), # Encode `Month` on X-axis
y="count()", # Aggregate records to show `count` for each Month
opacity=alt.condition(selection, alt.value(1), alt.value(0.2))
).add_params(
selection
).properties(
width=500
)

<ContextSensitiveImage src='img/tutorial_images/step_11.png' className='docs-image'/>
# Wrap VegaAltair chart object with PersistChart to enable persistence
PR.PersistChart(chart, df_name="av_ut3")
```

We will first create a new category, `Av_Season`, and add the three options using the new category popup. Next, we interactively select the months and assign them the appropriate phase.

<ContextSensitiveImage src='img/tutorial_images/step_12.png' className='docs-image'/>
Notice that the season is now part of our dataset, and we could facet our dataset based on the season for further analysis.
<ContextSensitiveImage src='img/tutorial_images/step_11.gif' className='docs-image'/>

Notice that the season is now part of our dataset, and we could facet our dataset based on the season for further analysis.

<ContextSensitiveImage src='img/tutorial_images/step_12.png' className='docs-image'/>

#### Equivalent pandas code

<ContextSensitiveImage src='img/tutorial_images/step_13.png' className='docs-image'/>
```py
av_ut3 = av_ut2.copy()

# Create a new column and set all values as `End`
av_ut3["Av_Season"] = "End"

# Assign `Start` to records for months 10, 11, 12
av_ut3.loc[av_ut3["Month"] >= 10, "Av_Season"] = "Start"

# Assign `Start` to records for months 10, 11, 12
av_ut3.loc[av_ut3["Month"] <= 3, "Av_Season"] = "Middle"
```

## The Persist Technique

Persist leverages the concept of interaction provenance as a shared abstraction between code and interactions within a notebook. Interaction provenance records all interactions leading to a particular point in the analysis. Each interaction is captured in the output of a code cell and documented in a provenance graph. This graph tracks the interactive analysis in real-time, supports navigation through the history, and allows branching off to explore alternative analysis paths.

Interactions recorded in the provenance graph are translated into data operations, updating the underlying dataframe. This updated dataframe is then used to refresh the output and is available as a new variable for further analysis.


## Caveats on using Vega-Altair and Persist

Persist works with Vega-Altair charts directly for the most part. Vega-Altair and Vega-Lite offer multiple ways to write a specification. However, Persist has certain requirements that need to be fulfilled.
Expand Down
1 change: 0 additions & 1 deletion docs-website/src/pages/community.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,3 @@ title: Community
Persist is meant to aid in the broader visualization and data science community. If you have any questions or feedback, consider joining our [Persist Slack channel](https://join.slack.com/t/slack-a521634/shared_invite/zt-2jobwmk39-IT~uYbDQyfDrXwBUwVxI7w).

Additionally, you can submit bugs and/or feature requests directly on our [Github](https://github.com/visdesignlab/persist).

Binary file modified docs-website/static/img/tutorial_images/step_1.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed docs-website/static/img/tutorial_images/step_10.png
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed docs-website/static/img/tutorial_images/step_11.png
Binary file not shown.
Binary file modified docs-website/static/img/tutorial_images/step_12.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs-website/static/img/tutorial_images/step_2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs-website/static/img/tutorial_images/step_3.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs-website/static/img/tutorial_images/step_6.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs-website/static/img/tutorial_images/step_8.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs-website/static/img/tutorial_images/step_9.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
60 changes: 30 additions & 30 deletions docs-website/yarn.lock
Original file line number Diff line number Diff line change
Expand Up @@ -2043,9 +2043,9 @@
integrity sha512-/kYRxGDLWzHOB7q+wtSUQlFrtcdUccpfy+X+9iMBpHK8QLLhx2wIPYuS5DYtR9Wa/YlZAbIovy7qVdB1Aq6Lyw==

"@types/express-serve-static-core@*", "@types/express-serve-static-core@^4.17.33":
version "4.19.1"
resolved "https://registry.yarnpkg.com/@types/express-serve-static-core/-/express-serve-static-core-4.19.1.tgz#57d34698bb580720fd6e3c360d4b2fdef579b979"
integrity sha512-ej0phymbFLoCB26dbbq5PGScsf2JAJ4IJHjG10LalgUV36XKTmA4GdA+PVllKvRk0sEKt64X8975qFnkSi0hqA==
version "4.19.2"
resolved "https://registry.yarnpkg.com/@types/express-serve-static-core/-/express-serve-static-core-4.19.2.tgz#ca09a37ffbdc66c584c305af0044b8ad3aa7b9ef"
integrity sha512-dPSEQElyVJ97BuGduAqQjpBocZWAs0GR94z+ptL7JXQJeJdHw2WBG3EWdFrK36b8Q6j8P4cXOMhgUoi0IIfIsg==
dependencies:
"@types/node" "*"
"@types/qs" "*"
Expand Down Expand Up @@ -2155,9 +2155,9 @@
"@types/node" "*"

"@types/node@*":
version "20.12.12"
resolved "https://registry.yarnpkg.com/@types/node/-/node-20.12.12.tgz#7cbecdf902085cec634fdb362172dfe12b8f2050"
integrity sha512-eWLDGF/FOSPtAvEqeRAQ4C8LSA7M1I7i0ky1I8U7kD1J5ITyW3AsRhQrKVoWf5pFKZ2kILsEGJhsI9r93PYnOw==
version "20.12.13"
resolved "https://registry.yarnpkg.com/@types/node/-/node-20.12.13.tgz#90ed3b8a4e52dd3c5dc5a42dde5b85b74ad8ed88"
integrity sha512-gBGeanV41c1L171rR7wjbMiEpEI/l5XFQdLLfhr/REwpgDy/4U8y89+i8kRiLzDyZdOkXh+cRaTetUnCYutoXA==
dependencies:
undici-types "~5.26.4"

Expand Down Expand Up @@ -2504,19 +2504,19 @@ ajv@^6.12.2, ajv@^6.12.5:
uri-js "^4.2.2"

ajv@^8.0.0, ajv@^8.9.0:
version "8.13.0"
resolved "https://registry.yarnpkg.com/ajv/-/ajv-8.13.0.tgz#a3939eaec9fb80d217ddf0c3376948c023f28c91"
integrity sha512-PRA911Blj99jR5RMeTunVbNXMF6Lp4vZXnk5GQjcnUWUTsrXtekg/pnmFFI2u/I36Y/2bITGS30GZCXei6uNkA==
version "8.14.0"
resolved "https://registry.yarnpkg.com/ajv/-/ajv-8.14.0.tgz#f514ddfd4756abb200e1704414963620a625ebbb"
integrity sha512-oYs1UUtO97ZO2lJ4bwnWeQW8/zvOIQLGKcvPTsWmvc2SYgBb+upuNS5NxoLaMU4h8Ju3Nbj6Cq8mD2LQoqVKFA==
dependencies:
fast-deep-equal "^3.1.3"
json-schema-traverse "^1.0.0"
require-from-string "^2.0.2"
uri-js "^4.4.1"

algoliasearch-helper@^3.13.3:
version "3.20.0"
resolved "https://registry.yarnpkg.com/algoliasearch-helper/-/algoliasearch-helper-3.20.0.tgz#1f31b11c0df9881d0086c8e7f64b4ccd66d78148"
integrity sha512-6EVhAmVug0+hdRHWbubF7hLHHhLoQ8NjLk6iS6d4k5chWawpS5EDexrF6Jx/hPZvUKIeNrzsbTpjAkcvrjNLHg==
version "3.21.0"
resolved "https://registry.yarnpkg.com/algoliasearch-helper/-/algoliasearch-helper-3.21.0.tgz#d28fdb61199b5c229714788bfb812376b18aaf28"
integrity sha512-hjVOrL15I3Y3K8xG0icwG1/tWE+MocqBrhW6uVBWpU+/kVEMK0BnM2xdssj6mZM61eJ4iRxHR0djEI3ENOpR8w==
dependencies:
"@algolia/events" "^4.0.1"

Expand Down Expand Up @@ -2870,9 +2870,9 @@ caniuse-api@^3.0.0:
lodash.uniq "^4.5.0"

caniuse-lite@^1.0.0, caniuse-lite@^1.0.30001587, caniuse-lite@^1.0.30001599:
version "1.0.30001621"
resolved "https://registry.yarnpkg.com/caniuse-lite/-/caniuse-lite-1.0.30001621.tgz#4adcb443c8b9c8303e04498318f987616b8fea2e"
integrity sha512-+NLXZiviFFKX0fk8Piwv3PfLPGtRqJeq2TiNoUff/qB5KJgwecJTvCXDpmlyP/eCI/GUEmp/h/y5j0yckiiZrA==
version "1.0.30001625"
resolved "https://registry.yarnpkg.com/caniuse-lite/-/caniuse-lite-1.0.30001625.tgz#ead1b155ea691d6a87938754d3cb119c24465b03"
integrity sha512-4KE9N2gcRH+HQhpeiRZXd+1niLB/XNLAhSy4z7fI8EzcbcPoAqjNInxVHTiTwWfTIV4w096XG8OtCOCQQKPv3w==

ccount@^2.0.0:
version "2.0.1"
Expand Down Expand Up @@ -2967,9 +2967,9 @@ chokidar@^3.4.2, chokidar@^3.5.3:
fsevents "~2.3.2"

chrome-trace-event@^1.0.2:
version "1.0.3"
resolved "https://registry.yarnpkg.com/chrome-trace-event/-/chrome-trace-event-1.0.3.tgz#1015eced4741e15d06664a957dbbf50d041e26ac"
integrity sha512-p3KULyQg4S7NIHixdwbGX+nFHkoBiA4YQmyWtjb8XngSKV124nJmRysgAeujbUVb15vh+RvFUfCPqU7rXk+hZg==
version "1.0.4"
resolved "https://registry.yarnpkg.com/chrome-trace-event/-/chrome-trace-event-1.0.4.tgz#05bffd7ff928465093314708c93bdfa9bd1f0f5b"
integrity sha512-rNjApaLzuwaOTjCiT8lSDdGN1APCiqkChLMJxJPWLunPAt5fy8xgU9/jNOchV84wfIxrA0lRQB7oCT8jrn/wrQ==

ci-info@^3.2.0:
version "3.9.0"
Expand Down Expand Up @@ -3662,9 +3662,9 @@ [email protected]:
integrity sha512-WMwm9LhRUo+WUaRN+vRuETqG89IgZphVSNkdFgeb6sS/E4OrDIN7t48CAewSHXc6C8lefD8KKfr5vY61brQlow==

electron-to-chromium@^1.4.668:
version "1.4.782"
resolved "https://registry.yarnpkg.com/electron-to-chromium/-/electron-to-chromium-1.4.782.tgz#726d7916f0f0a80524c2a2a19e5dceb60d35ce0e"
integrity sha512-JUfU61e8tr+i5Y1FKXcKs+Xe+rJ+CEqm4cgv1kMihPE2EvYHmYyVr3Im/+1+Z5B29Be2EEGCZCwAc6Tazdl1Yg==
version "1.4.784"
resolved "https://registry.yarnpkg.com/electron-to-chromium/-/electron-to-chromium-1.4.784.tgz#729a80154cee6ee9cb8bc3948af83431ab5cb394"
integrity sha512-9CZwh+sDrhDAeOEFh8s3PqwduzTyYIeYwZolc1b9ENAUt3ePu7R1sJSCWr/820ISssRxCJUyHI9Wb7j+0Uo1AA==

emoji-regex@^8.0.0:
version "8.0.0"
Expand Down Expand Up @@ -5149,9 +5149,9 @@ loader-utils@^2.0.0:
json5 "^2.1.2"

loader-utils@^3.2.0:
version "3.2.1"
resolved "https://registry.yarnpkg.com/loader-utils/-/loader-utils-3.2.1.tgz#4fb104b599daafd82ef3e1a41fb9265f87e1f576"
integrity sha512-ZvFw1KWS3GVyYBYb7qkmRM/WwL2TQQBxgCK62rlvm4WpVQ23Nb4tYjApUlfjrEGvOs7KHEsmyUn75OHZrJMWPw==
version "3.2.2"
resolved "https://registry.yarnpkg.com/loader-utils/-/loader-utils-3.2.2.tgz#dc154c005c65974dab413195c16cd246f545aecb"
integrity sha512-vjJi4vQDasD8t0kMpxe+9URAcgbSuASqoj/Wuk3MawTk97LYa2KfdHreAkd1G/pmPLMvzZEw7/OsydADNemerQ==

locate-path@^3.0.0:
version "3.0.0"
Expand Down Expand Up @@ -5261,9 +5261,9 @@ mdast-util-find-and-replace@^3.0.0, mdast-util-find-and-replace@^3.0.1:
unist-util-visit-parents "^6.0.0"

mdast-util-from-markdown@^2.0.0:
version "2.0.0"
resolved "https://registry.yarnpkg.com/mdast-util-from-markdown/-/mdast-util-from-markdown-2.0.0.tgz#52f14815ec291ed061f2922fd14d6689c810cb88"
integrity sha512-n7MTOr/z+8NAX/wmhhDji8O3bRvPTV/U0oTCaZJkjhPSKTPhS3xufVhKGF8s1pJ7Ox4QgoIU7KHseh09S+9rTA==
version "2.0.1"
resolved "https://registry.yarnpkg.com/mdast-util-from-markdown/-/mdast-util-from-markdown-2.0.1.tgz#32a6e8f512b416e1f51eb817fc64bd867ebcd9cc"
integrity sha512-aJEUyzZ6TzlsX2s5B4Of7lN7EQtAxvtradMMglCQDyaTFgse6CmtmdJ15ElnVRlCg1vpNyVtbem0PWzlNieZsA==
dependencies:
"@types/mdast" "^4.0.0"
"@types/unist" "^3.0.0"
Expand Down Expand Up @@ -7334,9 +7334,9 @@ [email protected], safe-buffer@>=5.1.0, safe-buffer@^5.1.0, safe-buffer@~5.2.0:
integrity sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==

sax@^1.2.4:
version "1.3.0"
resolved "https://registry.yarnpkg.com/sax/-/sax-1.3.0.tgz#a5dbe77db3be05c9d1ee7785dbd3ea9de51593d0"
integrity sha512-0s+oAmw9zLl1V1cS9BtZN7JAd0cW5e0QH4W3LWEK6a4LaLEA2OTpGYWDY+6XasBLtz6wkm3u1xRw95mRuJ59WA==
version "1.4.1"
resolved "https://registry.yarnpkg.com/sax/-/sax-1.4.1.tgz#44cc8988377f126304d3b3fc1010c733b929ef0f"
integrity sha512-+aWOz7yVScEGoKNd4PA10LZ8sk0A/z5+nXQG5giUO5rprX9jgYsTdov9qCchZiPIZezbZH+jRut8nPodFAX4Jg==

scheduler@^0.23.2:
version "0.23.2"
Expand Down

0 comments on commit 2205934

Please sign in to comment.