index.Rmd

---
title: "Strings and regex in R"
subtitle: "https://github.com/ecodatasci-tlv/strings-and-regex"
date: "`r Sys.Date()`"
output:
  xaringan::moon_reader:
    lib_dir: libs
    css: ["default", "custom-fonts.css"]
    nature:
      highlightStyle: github
      highlightLines: true
      ratio: '16:9'
      beforeInit: "https://platform.twitter.com/widgets.js"
---
```{r setup, include=FALSE}
library(xaringan)
library(knitr)
library(tidyverse)
options(tibble.print_min = 5)
conflicted::conflict_prefer("filter", "dplyr")
```

# When does it come up?

- data stored as notes
- non-uniformly formatted data
- filenames
- almost everywhere

---

# What are strings?

Strings (character types) = pretty much anything surrounded by quotes

**Double quotes** are the preferred style unless your text contains double quotes

.center[
![](https://media.giphy.com/media/xTiN0z1dTpeedhds8o/giphy.gif)
]

---

.center[.img-med[
![](https://media.giphy.com/media/HUjc0sbfgaInK/giphy.gif)
]]

### `"Keeps away the nargles."`  

--
### `'Luna whispered, "Keeps away the nargles."'`  

--
### `"Luna's eyes widened, and she whispered, \"Keeps away the nargles.\""`


---

# What's with the backslash?

.center[.img-big[
![](https://media.giphy.com/media/gCwXh8EHKKfMk/giphy.gif)
]]

---

# Escaped characters

Special characters that may have an alternate meaning

In our previous case: is `"` literally a double-quote (choop-chik) or is it marking a character string?  

--

Some other examples:

`\n` = new line  
`\t` = tab  
`\\` = backslash  

---

# Check your escapes with `writeLines()`

```{r}
writeLines("Nitwit! \n\tBlubber! \n\t\tOddment! \n\U1F9D9\t\t\tTweak!")
```

_You can also write emojis this way!_

---

# UTF-8 

Character-encoding system of choice but not fully supported by Windows (yet); this is how you can write in Hebrew, English, Chinese, and emojis in one sentence

Want to write in Hebrew? If regular Hebrew doesn't work, you can use the letter codes:

```{r}
writeLines("\U05DB\U05EA\U05D1") 
# write the letters-codes left-to-right but it prints right-to-left
```

---

class: inverse

# stringr

.center[
![](https://stringr.tidyverse.org/logo.png)

https://stringr.tidyverse.org  

_A consistent, simple and easy-to-use set of wrappers around the `stringi` package._
]

---

# General pattern

.center[.xlarge[
str_.gray[verb](.gray[string], .gray[...]) 
]] 

.large[
string = character string or vector

... = additional arguments include `pattern` to match or replacement string
]
---

# Functions for the day

.large[
- single-input functions: `str_verb(string)`
- multi-input functions: `str_verb(string, pattern, ...)`
- `*_all` variants
- `str_glue()` for interpolating strings
- `str_view()` for checking regex
]

If you're already familiar with the base R equivalents, check out [this vignette](https://stringr.tidyverse.org/articles/from-base.html) for the "translations"

---

# Let's get some data

```{r message = FALSE}
spells <- read_csv("data/spells.csv") # from the rcorpora pkg

(incantation <- spells$incantation[1:5])
```

.center[![](https://media.giphy.com/media/kSjGBipdCXdHW/giphy.gif)]

---

.center[.xlarge[
str_.gray[verb](.gray[string]) 
]] 

```{r}
# often a useful first step to avoid dealing with capitals
str_to_lower(incantation) 

str_length(incantation)
```

---

.center[.xlarge[
str_.gray[verb](.gray[string], .gray[pattern]) 
]] 

```{r}
str_detect(incantation, "o") # similar to grepl
str_subset(incantation, "o")
str_count(incantation, "o")
```

---

.center[.xlarge[str_.gray[verb]_all(.gray[string], .gray[...]) ]]

```{r}
str_extract_all(incantation, ".o")
```

---

class: exercise

# Your turn

How many `object`s are there among the effect descriptions?   
Replace `object` with a different noun. 

```{r}
spells
```

---

.center[.xlarge[
str_glue(.gray[...]) 
]] 

```{r}
# works similarly to `paste()` or `sprintf()`
str_glue('Hermione shouted, "{incantation}!"')
```

---

class: inverse, center, middle

.xlarge[regex]

## regular expressions

---

class: exercise

# How would you extract out amounts of money?

>"We'll bet 37 Galleons, 15 Sickles, 3 Knuts"  
"George and I invented them - 7 Sickles each, a bargain!"  
"True, both of them had paid 2 Sickles for a S.P.E.W. badge"  
"And 1000 Galleons prize money!"

--
Pseudocode:

```{r eval = FALSE}
money_types <- c("Galleons", "Sickles", "Knuts") 

sentences %>% 
  extract(number_before(money_types), money_types) 
```  

---

class: exercise

>"We'll bet 37 Galleons, 15 Sickles, 3 Knuts"  
"George and I invented them - 7 Sickles each, a bargain!"  
"True, both of them had paid 2 Sickles for a S.P.E.W. badge"  
"And 1000 Galleons prize money!"

>"I pull down about 100 sacks of galleons a year!"

--

Pseudocode:

```{r eval = FALSE}
money_types <- c("Galleons", "Sickles", "Knuts") %>% 
  to_lower_case() #<<

sentences %>% 
  to_lower_case() %>% #<<
  extract(closest_number_before(money_types)), money_types) #<<
```  

---

# regex helpers

.center[.xlarge[`str_view_all()`]] 

and `str_view()` are your friends

Other resources:

- [regex101](https://regex101.com) - interepret regex
- [regexplain](https://www.garrickadenbuie.com/project/regexplain/) - interpret and write regex (lots of cheatsheets)
- [rex](https://github.com/kevinushey/rex) package - "friendly" regex
- [rebus](https://cran.r-project.org/web/packages/rebus/rebus.pdf) package - "friendly" regex

---

# Key expressions 

_Note:_ There are often several ways of writing the same regex. For this presentation, I chose my favorite style, which I find the most flexible

`^` = start  
`$` = end  
`.` = anything  

```{r}
str_subset(c("football", "baseball", "ballroom"), "ball$")
```

---

.center[.xlarge[[.gray[one of]]]]

brackets = _one of_ the characters specified within the brackets (in the example, an `o`, `n`, `e`, space, or `f`)

`[a-z]` = any lower case letters  
`[0-9]` = any number from 0 to 9 (also `\\d`)

variations on the theme:  
`[09-]` = 0, 9, or -

---

class: exercise

# Guess the regex

.center[.xlarge[any vowel]]

--

.center[.xlarge[`[aeiou]`]]

---

class: exercise

# Guess the regex

.center[.xlarge[starts with a capital letter]]

--

.center[.xlarge[`^[A-Z]`]]

---

class: exercise

# Guess the regex

.center[.xlarge[a vowel and the two characters next to it]]

--

.center[.xlarge[`.[aeiou].`]]

---

# Numbers of things

`*` = 0 or more  
`+` = 1 or more  

`{n}` = exactly `n` number of times  
`{n,}` = at least `n` number of times  
`{n,m}` = between `n` and `m` times

---

class: exercise

# Guess the regex

.center[.xlarge[at least one number]]

--

.center[.xlarge[`[0-9]+`]]

---

class: exercise

# Guess the regex

.center[.xlarge[ends with `jpg` or `jpeg`]]

--

.center[.xlarge[`jp[e]{0,1}g`]]


---

class: exercise

# Guess the regex

.center[.xlarge[at least 2 #s in a row]]

--

.center[.xlarge[`[0-9]{2,}`]]

---

class: inverse, center, middle

# How to we say NOT?

---

class: center, middle

.img-big[
![](https://media.giphy.com/media/gbErpwcLlizvi/giphy.gif)
]

---

_Inside_ of brackets, a caret ("hat"/`^`) means **NOT**

_Outside_ of brackets and at the beginning of a string, it means **begins with**

.center[.img-big[![](https://media.giphy.com/media/FvJ4fmyI6gHLi/giphy.gif)]]

---

class: exercise

# Which is which?

.center[.xlarge[`^[a-z]` versus `[^a-z]`]]

---

class: inverse, center, middle

# Working in a dataframe

---

```{r}
spells
```

---

```{r}
spells %>% 
  mutate(effect = str_to_lower(effect)) #<<
```

---

```{r}
spells %>% 
  mutate(effect = str_to_lower(effect)) %>% 
  mutate(effect = str_split(effect, " ")) #<<
```

### Creates a _list-column_; supports vector results of different lengths

---

```{r}
spells %>% 
  mutate(effect = str_to_lower(effect)) %>% 
  mutate(effect = str_split(effect, " ")) %>% 
  unnest(effect) #<<
```

### _Note:_ this is very similar to the `unnest_tokens()` function in the [`tidytext`](https://www.tidytextmining.com/) package

---

```{r}
spells %>% 
  mutate(effect = str_to_lower(effect)) %>% 
  mutate(effect = str_split(effect, " ")) %>% 
  unnest(effect) %>% 
  count(effect, sort = TRUE) #<<
```

---

class: inverse, center, middle 

# A few more miscellaneous tricks

.x-large[![](https://media.giphy.com/media/54Yiwc8TSvoJO/giphy.gif)]

---

.center[.xlarge[(.gray[capture groups])]]

Extract groups within a pattern

```{r}
phone_numbers <- c("058 222 1234", "054-121 1221")

str_match(phone_numbers, "([0-9]+)[ -]([0-9]+)[ -]([0-9]+)")
# same as "(\\d+)[ -](\\d+)[ -](\\d+)"
```

---

.pull-left[
## look behinds `(?<=)` 

![](https://static.boredpanda.com/blog/wp-content/uploads/2016/10/newborn-baby-harry-potter-photo-shoot-kayla-glover-4.jpg)
]

.pull-right[
## look aheads `(?=)`

![](https://imagesvc.meredithcorp.io/v3/mm/image?url=https%3A%2F%2Fewedit.files.wordpress.com%2F2015%2F01%2Fharry-potter_510.jpg&w=400&c=sc&poi=face&q=85)
]

---

```{r}
str_extract("It does not do to dwell on dreams and forget to live.",
            "(?<=and )[A-z]+") #<<

str_extract("It does not do to dwell on dreams and forget to live.",
            "[A-z]+(?= and)") #<<
```

---

# Bonus question

How do you extract the currency conversions from this text?

```{r}
text <- c("There are 29 Knuts in 1 silver Sickle",
          "and there are 493 Knuts in 1 golden Galleon.")
```

```{r include = FALSE}
tibble(n_knuts = str_extract(text, "[0-9]+(?= Knuts)"),
       comparison = str_extract(text, "(?<=in )[0-9]+"),
       unit = str_extract(text, "(?<=in )[0-9 ]+[A-z ]+"))
```

--

Shortcut if you only care about the first number...

```{r}
parse_number(text)
```

---

# Summary

- strings/characters = things that are quoted in R
- backslashes are used for "escape characters" (`\n`, `\\`)
- use `writeLines()` to show how strings will display
- `stringr` provides a simple, consistent interface to work with strings in R
- regular expressions describe the logic of a pattern in text
- check regex in R with `str_view()`/`str_view_all()`
- use `unnest()` to expand list-columns

Cheatsheets: [Basic regular expressions in R](https://www.rstudio.com/wp-content/uploads/2016/09/RegExCheatsheet.pdf), [Working with strings with stringr](https://resources.rstudio.com/rstudio-cheatsheets/stringr-cheat-sheet)

Santa Barbara Eco-Data-Science [text workshop materials](https://github.com/eco-data-science/text_workshop) (covers pdftools, sentiment analysis, etc.)

---

# You are now ready to face the world of strings and regex

.center[.xlarge[
![](https://media.giphy.com/media/ffynNaSYx2yTC/giphy.gif)]]