```{r}
#| label: packages
#| echo: false
#| message: false

library(tidyverse)
library(webexercises)
```

## Rmarkdown

Last week, you learnt how to use R scripts to save your code.

Keeping track of the code you use for data analysis is a very important aspect of research project managing: not only the code is there if you need to rerun it later, but it allows your data analysis to be **reproducible** (i.e., it can be reproduced by you or other people in such a way that starting with the same data and code you get to the same results).

::: callout-tip
#### Reproducible research

Research is **reproducible** when the same data and same code return the same results.
:::

R scripts are great for writing code, and you can even document the code (add explanations or notes) with comments (i.e. lines that start with `#`).

But for longer text or complex data analysis reports, R scripts can be a bit cumbersome.

A solution to this is using Rmarkdown files (they have the `.Rmd` extension).

### Code... and text!

Rmarkdown is a file format that allows you to mix code and formatted text in the same file.

This means that you can write **dynamic reports** using Rmd files: dynamic reports are just like analysis reports (i.e. they include formatted text, plots, tables, code output, code, etc...) but they are **dynamic** in the sense that if, for example, data or code changes, you can just rerun the report Rmd file and all code output (plots, tables, etc...) is updated accordingly!

You can watch this short video for a quick tour.



::: callout-tip
#### Dynamic reports in R Markdown

**R Markdown** is a file type with extension `.Rmd` in which you can write formatted text and code together.

R Markdown can be used to generate **dynamic reports**: these are files that are generated automatically from the R Markdown source, ensuring data and results in the report are always up to date.
:::

### Formatting text

R comments in R scripts cannot be formatted (for example, you can't make bold or italic texts).

Text in Rmd files can be fully formatted using a simple but powerful **mark-up language** called [markdown](https://www.markdownguide.org).

You don't have to learn markdown all in one go, so I encourage you to just learn it bit by bit, in your time. You can look at the the [Markdown Guide]() for an in-depth intro and/or dive in the [Markdown Tutorial](https://www.markdowntutorial.com) for a hands-on approach.

A few quick pointers (you can test them in the [Markdown Live Preview](https://markdownlivepreview.com)):

-   Text can be made italics by enclosing it between single stars: `*this text is in italics*`.

-   You can make text bold with two stars: `**this text is bold!**`.

-   Headings are created with `#`: `# This is a level-1 heading`. `## This is a level-2 heading`.

::: callout-tip
#### Mark-up, Markdown

A **mark-up language** is a text-formatting system consisting of symbols or keywords that control the structure, formatting or relationships of textual elements. The most common mark-up languages are HTML, XML and TeX.

**Markdown** is a simple yet powerful mark-up language.
:::

### Create an Rmd file

::: callout-warning
#### Important

When working through these tutorials, always **make sure you are in the course RStudio Project** you just created.

You know you are in an RStudio Project because you can see the name of the Project in the top-right corner of RStudio, next to the light-blue cube icon.

If you see `Project (none)` in the top-right corner, that means **you are not** in an RStudio Project.

To make sure you are in the RStudio project, you can open the project by going to the project folder in File Explorer (Win) or Finder (macOS) and double click on the `.Rproj` file.
:::

To create a new Rmd file, just click on the `New file` button, then `R Markdown...`. (If you are asked to install/update packages, do so.)

![](/img/new-rmd.png)

A window will open. Add a title of your choice and your name, then click `OK`.

![](/img/new-rmd-window.png)

A new Rmd file will be created and will open in the File Editor panel in RStudio.

**Note** that creating an Rmd file does not automatically saves it on your computer. To do so, either use the keyboard short-cut `CMD+S`/`CTRL+S` or click on the floppy disk icon in the menu below the file tab.

![](/img/save-rmd.png)

Save the file inside the `code/` folder with the following name: `tutorial-w03.Rmd`.

Remember that all the files of your RStudio project don't live inside RStudio but on your computer. So you can always access them from the Finder or File Explorer!

### Parts of an Rmd file

An Rmd file usually has three main parts:

-   The preamble (green).

-   Code chunks (orange).

-   Text (blue).

![](/img/rmd-parts.png)

Each Rmd file has to start with a preamble, but you can include as many code chunks and as much text as you wish, in any order.

::: callout-tip
#### R Markdown: Preamble

The **preamble** of an `.Rmd` file contains a list of `key: value` pairs, used to specify settings or document info like the `title` and `author`.

Preambles start and end with three dashes `---`.
:::

::: callout-tip
#### R Markdown: Code chunks

**Code chunks** start and end with three back-ticks ```` ``` ```` and they contain code.

Settings can be specified between curly brackets `{}` on the first ```` ``` ```` line: for example `{r chunk-name, eval=FALSE}`. `r` indicates that the code is R code.
:::

### Render Rmd files to HTML

You can render an Rmd file into a nicely formatted HTML file.

To render an Rmd file, just click on the **Knit** button and an HTML file will be created and saved in the same location of the Rmd file.

![](/img/knit.png)

It will also be shown in the Viewer pane (like in the picture below) or in a new window (you can set this option in the RStudio preferences \> R Markdown \> Basics \> Show output preview in...).

![](/img/rendered.png)

Rendering Rmd files is not restricted to HTML, but also PDFs and even Word documents!

This is very handy when you are writing an analysis report you need to share with others.

::: callout-tip
#### R Markdown: Rendering

Rmd files can be **rendered** into other formats, like HTML, PDF and Word documents.
:::

### How to run code

The first code chunk in an Rmd file is a special one: it's called `setup` and it's where you attach all the packages needed for running the code in the file. (You can find the name of a code chunk---or define one yourself!---in the first line of a code block: ```` ```{r chunk-name} ````). The first code chunk of an Rmd file should always be named `setup`.

The `setup` code chunk also has special code from the knitr package. The knitr package is what allows you to run R code from Rmd files.

The line `knitr::opts_chunk$set(echo = TRUE)` sets the option `echo` to `TRUE`. The `echo` option let's you decide whether you want the R code printed (echoed) in the rendered Rmd file. When `echo = TRUE` the code is printed, when `echo = FALSE` the code is not printed.

Now go ahead and add `library(tidyverse)` in the line below the `knitr::opts_chunk` line.

::: callout-note
#### Running code

To run the code of a code chunk you can use different approaches:

-   Click on the green arrow to the right of the code chunk to run the entire code chunk.

-   Alternatively, place the text cursor inside the code chunk and press `CMD/CTRL + SHIFT + ENTER`.

-   You can also run line by line as you do in an R script by placing the text cursor on the line you want to run and press `CMD/CTRL + ENTER`.
:::

Now delete everything after the `setup` code chunk and write a new level-2 heading after the chunk with the name "Plotting basics". (Remember to leave an empty line between the chunk and the heading.)

![](/img/rmd-setup.png)

You will use this Rmd file to write text and code for this tutorial. To insert a new code chunk, you can click on the `Insert a new code chunk` button and then select `R`, or you can press `OPT/ALT + CMD/CTRL + I`.

![](/img/rmd-new-chunk.png)

A new R code chunk will be inserted at the text cursor position.

## Plotting basics

Plotting data in R is easy once you understand the basics.

### Graphic systems

In R, you can create plots using different systems.

-   Base R.
-   lattice.
-   ggplot2.
-   more...

In this course you will learn how to use the ggplot2 system, but before we dive in, let's have a look at the base R plotting system too.

### Base R plotting function

Let's create two vectors, `x` and `y` and plot them. For now, run these in the Console (not in the code chunk we just created).

```{r}
#| label: scatter

x <- 1:10
y <- x^3

plot(x, y)
```

Easy!

Now let's add a few more things.

```{r}
#| label: line

plot(x, y, type = "l", col = "purple", lwd = 3, lty = "dashed")
```

With plots as simple as this one, the base R plotting system is sufficient, but to create more complex plots (which is virtually always the case), base R gets incredibly complicated.

Instead we can use the tidyverse package [ggplot2](https://ggplot2.tidyverse.org). ggplot2 works well with the other tidyverse packages and it follows the same principles, so it is convenient to use it for data visualisation instead of base R!

## Your first ggplot2 plot

The tidyverse package [ggplot2](https://ggplot2.tidyverse.org) provides users with a consistent set of functions to create captivating graphics.

::: callout-warning
To be able to use the functions in a package, you first need to **attach the package**. We have already attached the `library(tidyverse)` packages, among which there is ggplot2, so you don't need to do anything else.
:::

### The `polite` data

We will use the `polite` data to learn the basics of plotting using ggplot.

The data comes from the paper by Winter and Grawunder, 2020. *The phonetic profile of Korean formal and informal speech registers*. DOI: [10.1016/j.wocn.2012.08.006](https://doi.org/10.1016/j.wocn.2012.08.006). Read the abstract to familiarise yourself with the study.

To download the file with the data right-click on the following link and download the file: [polite.csv](../data/polite.csv). (Note that tutorial files are also linked in [Course content](../content.qmd)). Remember to save the file in `data/` in the course project folder.

With Rmd files, we can use relative paths to read data. For relative paths to work relative to the project working directory, you need to install the [here](https://here.r-lib.org) package (remember how to install packages?) and **add the following line in the setup chunk**, just after the first line of code.

``` r
knitr::opts_knit$set(root.dir = here::here())
```

(Note that it's `knitr::opts_knit` rather than `knitr::opts_chunk`).

Now rerun the `setup` code chunk to set the new option.

This will set the working directory for the Rmd file to the project working directory.

::: callout-warning
**Make sure that you [always]{.underline} include that** `knitr::opts_knit()` **line in the `setup` block**, or R might get confused about where the working directory is and not be able to find the data.
:::

Finally, **create a new code chunk, add the following line of code and then run the code chunk to read the data**.

```{r}
#| label: polite
#| message: false

polite <- read_csv("data/polite.csv")

```

In this tutorial we will use the following columns:

-   `f0mn`: mean f0 (fundamental frequency).

-   `H1H2`: difference between H2 and H1 (second and first harmonic).

-   `gender`: speaker's gender.

-   `musicstudent`: whether the speaker is a music student (`yes`) or not (`no`).

::: callout-note
#### Quiz 1

```{r}
#| label: quiz-1
#| results: asis
#| echo: false

opts_1 <- c(
   "The current working directory.",
   answer = "The project directory.",
   "The data directory."
)

cat("What does `here::here()` return?", longmcq(opts_1))
```

::: {.callout-important collapse="true"}
#### Explanation

The answer `The current working directory.` is wrong because if you manually set the working directory with `setwd(),` `here::here()` will still return the project directory!
:::
:::

### A basic plot

These are the minimum constituents of a ggplot2 plot.

::: callout-tip
## ggplot basics

-   The `data`: you have to specify the data frame with the data you want to plot.

-   The `mapping`: the mapping tells ggplot how to map data columns to parts of the plot like the axes or groupings within the data. These parts are called *aesthetics*, or `aes` for short.
:::

You can specify the data and mapping with the `data` and `mapping` arguments of the `ggplot()` function.

Note that the `mapping` argument is always specified with `aes()`: `mapping = aes(…)`.

In the following bare plot, we are just mapping the mean f0 (`f0`) to the *x*-axis and the H2-H1 difference to the *y*-axis, from the `polite` data frame.

**Create a new code chunk, copy the following code and run it.** From this point on I will assume you'll create a new code chunk and run the code yourself, without explicit instructions.

```{r}
#| label: bare-plot

ggplot(
  data = polite,
  mapping = aes(x = f0mn, y = H1H2)
)
```

Not much to see here: just two axes!

::: callout-note
#### Quiz 2

```{r}
#| label: quiz-2
#| results: asis
#| echo: false

cat("Is the following code correct? Justify your answer.", torf(FALSE))

```

``` r
ggplot(
  data = polite,
  mapping = c(x = total_duration, y = articulation_rate)
)
```
:::

### Let's add geometries

Nice, but we are missing the most important part: showing the data!

Data is represented with **geometries**, or `geom`s for short. `geom`s are added to the base ggplot with functions whose names all start with `geom_`.

::: callout-tip
#### Geometries

**Geometries** are plot elements that show the data through geometric shapes.

Different geometries are added to a ggplot using one of the `geom_*()` functions.
:::

For this plot, you want to use `geom_point()`. This geom simply adds point to the plot based on the data in the `polite` data frame.

To add `geom`s to a plot, you write a `+` at the end of the `ggplot()` command and include the geom on the next line. For example:

```{r}
#| label: first-plot

ggplot(
  data = polite,
  mapping = aes(x = f0mn, y = H1H2)
) +
  geom_point()
```

This type of plot, with two continuous axes and data represented by points, is called a **scatter plot**.

::: callout-tip
#### Scatter plot

A **scatter plot** is a plot with two numeric axes and points indicating the data. It is used when you want to show the relationship between two numeric variables.

To create a scatter plot, use the `geom_point()` geometry.
:::

When writing your results section, you could describe the plot this way:

> Figure 1 shows a scatter plot of mean f0 on the *x*-axis and H1-H2 difference on the *y*-axis. The plot suggest an overall negative relationship between mean f0 and H1-H2 difference.

::: callout-warning
Note that using the `+` is a quirk of `ggplot()`. The idea behind it is that you start from a bare plot and you **add** (`+`) layers of data on top of it. This is because of the philosophy behind the package, called the [Layered Grammar of Graphics](http://vita.had.co.nz/papers/layered-grammar.html).
:::

### Function arguments

Note that the `data` and `mapping` arguments don't have to be named explicitly (with `data =` and `mapping =`) in the `ggplot()` function, since they are obligatory and they are specified in that order.

```{r}
#| label: named-arguments
#| warning: false

ggplot(
  polite,
  aes(x = f0mn, y = H1H2)
) +
  geom_point()
```

In fact, you can also leave out `x =` and `y =`.

```{r}
#| label: named-arguments-2

ggplot(
  polite,
  aes(f0mn, H1H2)
) +
  geom_point()
```

Try running `?ggplot` in the Console to see the arguments of the function and the order they appear in.

::: callout-note
#### Quiz 3

```{r}
#| label: quiz-3
#| results: asis
#| echo: false

opts_3 <- c(
  "`ggplot(polite, aes(H1H2, f0mn)) + geom_point()`",
  answer = "`ggplot(polite, aes(y = H1H2, x = f0mn)) + geom_point()`",
  "`ggplot(polite, aes(y = f0mn, x = H1H2)) + geom_point()`"
)

cat("**Which of the following will produce the same plot as the one above?** Reason through it first without running the code, then run all of these to check whether they look the way you expected.", longmcq(opts_3))
```

::: {.callout-tip collapse="true"}
#### Hint

When specifying arguments, the order matters when not using the argument names.

So this `aes(a, b)` is different from `aes(b, a)`.

But this `aes(y = b, x = a)` is the same as `aes(a, b)`.
:::
:::

## Bar charts

Another common type of plot is the **bar chart**.

::: callout-tip
#### Bar charts

**Bar charts** are useful when you are counting things. For example:

-   Number of verbs vs nouns vs adjectives in a corpus.

-   Number of languages by geographic area.

-   Number of correct vs incorrect responses.

The bar chart geometry is `geom_bar()`.
:::

In the following example, we will be counting the number of languages by their endangerment status.

### Do you understand me?

There are thousands of languages in the world, but most of them are losing speakers, and some are already no longer spoken. The endangerment status of a language in the data is on a scale from `not endangered` (languages with large populations of speakers) through `threatened`, `shifting` and `nearly extinct`, to `extinct` (languages that have no living speakers left).

The `glot_status` data frame contains the endangerment status for 7,845 languages from [Glottolog](https://glottolog.org). To download the file with the data right-click on the following link and download the file: [glot_status.rds](../data/glot_status.rds). (Note that tutorial files are also linked in the [Syllabus](../content.qmd)). Remember to save the file in `data/` in the course project folder.

This time the file is not a `.csv` file but an `.rds` file. `.rds` files are files that can save any R object. They are commonly used to save data frames, like in this case.

::: callout-tip
#### Rds files

`.Rds` files are a type of R file which can store any R object and save it on disk.

R objects can be saved to an `.Rds` file with the `saveRDS()` function and they can be read with the `readRDS()` function.
:::

To read an `.rds` file you need the `readRDS()` function.

```{r}
#| label: glot-status

glot_status <- readRDS("data/glot_status.rds")

```

Here's what the data frame looks like.

```{r}
#| label: glot-status-view

glot_status
```

For us today, the relevant columns are:

-   `Name`: the language name.

-   `status`: the endangerment status of the language.

-   `Macroarea`: the geographic macro-area the language is spoken in.

### Create a bar chart

To create a bar chart, use the `geom_bar()` geometry.

::: callout-warning
## Bar chart axes

In a simple bar chart, **you only need to specify one axis, the *x*-axis**, in the aesthetics `aes()`.

This is because the counts that are placed on the *y*-axis are calculated by the `geom_bar()` function under the hood.

This quirk is something that confuses many new learners, so make sure you internalise this.
:::

Go ahead and complete the following code to create a bar chart.

```{r}
#| label: status-bar-ex
#| eval: false

glot_status %>%
  ggplot(aes(x = status)) +
  ...
```

Note how we're using `%>%` to pipe the `glot_status` data frame into the `ggplot()` function. This works because `ggplot()`'s first argument is the data, and piping is a different way of providing the first argument to a function.

As mentioned above, the counting for the *y*-axis is done automatically. R looks in the `status` column and counts how many times each value in the column occurs in the data frame.

If you did things correctly, you should get the following plot.

```{r}
#| label: status-bar
#| echo: false

glot_status %>%
  ggplot(aes(x = status)) +
  geom_bar()
```

A description of the plot:

> The number of languages by endangered status is shown as a bar chart in Figure 2. Among the languages that are endangered, excluding those that are extinct, the majority are either threatened or shifting.

### Stacked bar charts

A special type of bar charts are the so-called stacked bar charts.

::: callout-tip
#### Stacked bar chart

A **stacked bar chart** is a bar chart in which each contains a "stack" of shorter bars, each indicating the counts of some sub-groups.

This type of plot is useful to show how counts of something vary depending on some other grouping (in other words, when you want to count the occurrences of a categorical variable based on another categorical variable). For example:

-   Number of languages by endangerment status, grouped by geographic area.

-   Number of infants by head-turning preference, grouped by first language.

-   Number of past vs non-past verbs, grouped by verb class.
:::

To create a stacked bar chart, you just need to add a new aesthetic mapping to `aes()`: `fill`. The `fill` aesthetic lets you fill bars or areas with different colours depending on the values of a specified column.

Let's make a plot on language endangerment by macro-area.

Complete the following code by specifying that `fill` should be based on `status`.

```{r}
#| label: status-stacked-ex
#| eval: false

ggplot(
  glot_status,
  aes(x = Macroarea, ...)
) +
  geom_bar()
```

You should get the following.

```{r}
#| label: status-stacked
#| echo: false

ggplot(
  glot_status,
  aes(x = Macroarea, fill = status)
) +
  geom_bar()
```

A write-up example:

> Figure 3 shows the number of languages by geographic macroarea, subdivided by endangerment status. Africa, Eurasia and Papunesia have substantially more languages than the other areas.

::: callout-note
#### Quiz 4

What is wrong in the following code?

``` r
gestures %>%
  ggplot(aes(x = status), fill = Macroarea) +
  geom_bar()
```
:::

In the plot above it is difficult to assess whether different macroareas have different proportions of endangerment. This is because the overall number of languages per area differs between areas.

A solution to this is to plot **proportions** instead of raw counts.

You could calculate the proportions yourself, but there is a quicker way: using the `position` argument in `geom_bar()`.

You can plot proportions instead of counts by setting `position = "fill"` inside `geom_bar()`, like so:

```{r}
#| label: status-stacked-2
#| echo: false

ggplot(
  glot_status,
  aes(x = Macroarea, fill = status)
) +
  geom_bar(position = "fill")
```

The plot now shows proportions of languages by endangerment status for each area separately. (Note that the *y*-axis label is still "count" but it is in fact proportions; you will learn how to change labels next week).

With this plot it is easier to see that different areas have different proportions of endangerment. In writing:

> Figure 4 shows proportions of languages by endangerment status for each macroarea. Australia, South and North America have a substantially higher proportion of extinct languages than the other areas. These areas also have a higher proportion of near extinct languages. On the other hand, Africa has the greatest proportion of non-endangered languages followed by Papunesia and Eurasia, while North and South America are among the areas with the lower proportion, together with Australia which has the lowest.

## Faceting and panels

Sometimes we might want to separate the data into separate panels.

We can achieve that easily using **faceting**.

### Polite again

Let's reproduce one of the plots from above, but this time let's use the `colour` aesthetic to colour the points by `gender`.

```{r}
#| label: polite-gender
#| warning: false

polite %>%
  ggplot(aes(f0mn, H1H2, colour = gender)) +
  geom_point()
```

### Does being a music student matter?

That looks great, but we want to know if being a music student has an effect on the relationship of `f0mn` and `H1H2`.

In the plot above, the aesthetics mappings are the following:

-   `f0mn` on the *x*-axis.
-   `H1H2` on the *y*-axis.
-   `gender` as colour.

How can we separate data further depending on whether the participant is a music student or not (`musicstudent`)?

We can create panels using `facet_grid()`. This function takes lists of variables to specify panels in rows and/or columns.

::: callout-note
#### Faceting

**Faceting** a plot allows to split the plot into multiple panels, arranged in rows and columns, based on one or more variables.

To facet a plot, use the `facet_grid()` function.
:::

The syntax is a bit strange. Check the documentation with `?facet_grid()` (especially the examples) to see how it works and then complete the code below and run it to produce a plot with two column panels for `musicstudent`.

```{r}
#| label: polite-facet-ex
#| eval: false

polite %>%
  ggplot(aes(f0mn, H1H2, colour = gender)) +
  geom_point() +
  facet_grid(...)
```

This should be the result.

```{r}
#| label: polite-facet
#| echo: false
#| warning: false

polite %>%
  ggplot(aes(f0mn, H1H2, colour = gender)) +
  geom_point() +
  facet_grid(~ musicstudent)
```

> The figure shows mean f0 and H1-H2 difference as a scatter plot. The two panels indicate whether the participant was a student of music. Within each panel, the participant's gender is represented by colour (red for female and blue for male). Male participants tend to have higher H1-H2 differences and lower mean f0 than females. From the plot it can also be seen that there is greater variability in H1-H2 difference in female music students compared to female non-music participants. Within each group of gender by music student there does not seem to be any specific relation between mean f0 and H1-H2 difference.

## Render your Rmd file!

Now that you have done all of this hard work, why don't you try and render the Rmd file you've been working on to an HTML file?

Go ahead, click on the "Knitr" button and if everything works fine you should see a rendered HTML file in a second!

Note that you will be asked to render your Rmd files for the assessments, so I recommend you try this out now.

## Summary

That's all for this week!

::: {.callout-note appearance="minimal"}
**R Markdown**

-   **R Markdown** can be used to create dynamic and reproducible reports.

-   **Mark-up languages** are text-formatting systems that specify text formatting and structure using symbols or keywords. Markdown is the mark-up language R Markdown is based on.

-   The main parts of an `.Rmd` file are the preamble, text and code chunks.

**Plotting**

-   **ggplot2** is a plotting package from the tidyverse.

    -   To create a basic plot, you use the `ggplot()` function and specify `data` and `mapping`.

    -   The `aes()` function allows you to specify aesthetics (like axes, colours, ...) in the `mapping` argument.

    -   Geometries map data values onto shapes in the plot. All geometry functions are of the type `geom_*()`.

-   **Scatter plots** are created with `geom_point()` and can be used with two numeric variables.

-   **Bar charts** are created with `geom_bar()` and can be used to show the counts of different levels of a categorical variable.

-   **Stacked bar charts** are created by specifying the `fill` aesthetic and can be used to show counts grouped by a second categorical variable.

-   **Stacked bar charts showing proportions** can be created with the argument `position = "fill"` in `geom_bar()`.

-   **Faceting** allows you to split the plot in different panels, based on other variables in the data. Faceting is achieved with `facet_grid()`.
:::

Next week, we will start learning about statistical modelling and while doing that you will also learn how to create density and violin plots and how to customise legends, titles and labels!
