I Dataset Description

Here is a brief description of the datasets that are used in this book.

I.1 Alcohol disorders

In the book repo: alcohol-disorders.csv.

Share of males and females, suffering from alcohol use disorders (percentage of population). Alcohol dependence is defined by the International Classification of Diseases as the presence of three or more indicators of dependence for at least a month within the previous year. This is given as the age-standardized prevalence which assumes a constant age structure allowing for comparison by sex, country and through time.

IHME, Global Burden of Disease Study (2019) – processed by Our World in Data.
Dowloaded from OWiD

Variables:

Entity: country, only Argentina, Kenya, Taiwan, Ukraine and the U.S. are included.
Code: 3-letter country code
Year: 2015–2019 (only a subset of the original)
disordersM: number of cases of alcohol use disorders per 100 people, in males, age-standardized
disordersF: number of cases of alcohol use disorders per 100 people, in females, age-standardized,
population: Population (historical estimates),

Example:

read_delim("data/alcohol-disorders.csv") %>%
   sample_n(4)

## # A tibble: 4 × 6
##   country Code   Year disordersM disordersF population
##   <chr>   <chr> <dbl>      <dbl>      <dbl>      <dbl>
## 1 Kenya   KEN    2018      0.752      0.662   49953300
## 2 Kenya   KEN    2015      0.747      0.654   46851496
## 3 Kenya   KEN    2017      0.742      0.658   48948140
## 4 Taiwan  TWN    2016      0.889      0.261   23594476

I.2 Amazon reviews

Reviews of beauty products from Amazon, a random sample of 20 from a larger dataset. Originally downloaded from https://nijianmo.github.io/amazon/index.html, the originals are in json.

Columns:

date: date of the review
verified:
summary: (user provided) summary of the review
review: user’s review of the product
rating: 1-5, 1 being the lowest and 5 the highest rating.

Example:

reviews <- read_delim("data/amazon-reviews.csv.bz2")
reviews %>%
   head(2)

## # A tibble: 2 × 5
##   date       verified summary              
##   <date>     <lgl>    <chr>                
## 1 2014-07-08 TRUE     Good color don't stay
## 2 2014-03-18 TRUE     Light Weight         
##   review                                                                                    
##   <chr>                                                                                     
## 1 Polish chips very easily  Love the color.                                                 
## 2 This is a light weight conditioner as described.  It is not deep conditioning if you have…
##   rating
##    <dbl>
## 1      2
## 2      3

I.3 Any drinking

In the book repo: alcohol-disorders.csv.

Estimates for alcohol consumption for patterns in 10 randomly selected U.S. counties for 2008 and 2012. The numbers represent prevalence (= drinkers/adult population) of drinking in each county. The data comes from the Drinking-patterns study. at the Institute for Health Metrics and Evaluation website, and was downloaded from healthdata.org. For “any drinking”, drinkers are those who consumed at least one drink of any alcoholic beverage in the past 30 days.

Estimates are provided for males and females separately, as well as both sexes combined.

Columns: * state: name of the state * location: county name. For state level observations it is the same as state * both_sexes_2008, females_2008, males_2008: the corresponding drinking rates for everyone, females and males. It includes years 2008 and 2012.

Example:

read_delim("data/any-drinking.csv") %>%
   sample_n(3)

## # A tibble: 3 × 8
##   state     location        both_sexes_2008 females_2008 males_2008 both_sexes_2012
##   <chr>     <chr>                     <dbl>        <dbl>      <dbl>           <dbl>
## 1 Florida   DeSoto County              42.9         36.3       49.7            45.8
## 2 Wisconsin Marathon County            71.5         67         76.1            68.5
## 3 Virginia  Surry County               41.1         31.3       51.3            41.8
##   females_2012 males_2012
##          <dbl>      <dbl>
## 1         40         51.8
## 2         65.2       71.8
## 3         32.6       51.3

I.4 Babynames

R package babynames contains a dataset babynames. It includes all names, given to babies born in the U.S. between 1880-2017. Only names that were given to at least 5 babies each year for each sex. Data originates from the U.S. Social Security Administration.

You can load it with library(babynames), that loads a single data frame babynames.

Example:

library(babynames)
babynames %>%
   sample_n(3)

## # A tibble: 3 × 5
##    year sex   name        n       prop
##   <dbl> <chr> <chr>   <int>      <dbl>
## 1  1992 F     Khrysta     6 0.00000299
## 2  2004 F     Ameila      8 0.00000397
## 3  1964 F     Stella    560 0.000286

Variables:

year: 1880-2017
name: the name
sex: “F” or “M”
n: how many babies got this name (withing year/sex)
prop: proportion of babies who got this name in the given year (within year/sex).

I.5 Country-concept similarity

In the book repo: country-concept-similarity.csv.bz2. This dataset shows the similarity between country names and a set of different words, and it is calculated based on texts that were scraped from internet around 2015. The dataset looks like

similarity <- read_delim("data/country-concept-similarity.csv.bz2")
similarity %>%
   head(2)

## # A tibble: 2 × 12
##   country     terrorism nuclear  trade battery  regime volcano  palm    fir  flood drought
##   <chr>           <dbl>   <dbl>  <dbl>   <dbl>   <dbl>   <dbl> <dbl>  <dbl>  <dbl>   <dbl>
## 1 aruba          0.0891  -0.011 0.0504 -0.01   -0.0356   0.166 0.293 0.0965 0.0158  0.0581
## 2 afghanistan    0.447    0.220 0.109   0.0578  0.180    0.129 0.116 0.129  0.159   0.160 
##   mountain
##      <dbl>
## 1    0.107
## 2    0.161

One can see that “Afghanistan” and “terrorism” are much more similar (similarity 0.447) than e.g. “Afghanistan” and “trade” (similarity 0.109). We do not go into details here about how the similarity is measured, but broadly, it means how frequently are these words used in a similar context as the corresponding country names.

I.6 Covid in Scandinavia

In the book repo: covid-scandinavia.csv.bz2

Data dowloaded from github data

Extracted a subset of Scandinavian countries, only national level, only deaths, confirmed cases. Added per capita counts and daily growth numbers. Example:

read_delim("data/covid-scandinavia.csv.bz2") %>%
   sample_n(4)

## # A tibble: 4 × 11
##   code2 country state date       type      count lockdown   population   countPC growth
##   <chr> <chr>   <lgl> <date>     <chr>     <dbl> <date>          <dbl>     <dbl>  <dbl>
## 1 SE    Sweden  NA    2020-02-11 Deaths        0 NA           10377781 0              0
## 2 NO    Norway  NA    2020-08-25 Deaths      264 2020-03-12    5384576 0.0000490      0
## 3 NO    Norway  NA    2020-07-06 Deaths      251 2020-03-12    5384576 0.0000466      0
## 4 NO    Norway  NA    2020-06-20 Confirmed  8742 2020-03-12    5384576 0.00162       16
##     growthPC
##        <dbl>
## 1 0         
## 2 0         
## 3 0         
## 4 0.00000297

Variables:

code2: 2-letter country code
country: country name
state: federal state, just NA in case of Scandinavian countries
date: date of the count
type: count type: Confirmed/Deaths
count: how many persons have confirmed covid/died
lockdown: whether the country under a major lockdown (1/0)
population: country population (only one number)
countPC: count per capita
growth: growth in count
growthPC: growth in count per capita

I.7 CS-GO

Dataset about CS-GO (video game) reviews: each line is a review. It is scraped from Steam website by mulhod, see the original repo at GitHub. The dataset is not really documented, but you can guess based on the column names.

Sample:

read_delim("data/csgo-reviews.csv.bz2") %>%
   sample_n(3)

## # A tibble: 3 × 8
##   rating      nHelpful nFunny nScreenshots date                  hours nGames nReviews
##   <chr>          <dbl>  <dbl>        <dbl> <chr>                 <dbl>  <dbl>    <dbl>
## 1 Recommended        6      0            0 Dec 16, 2013, 10:40AM 3101.      0        0
## 2 Recommended        3      1            0 Nov 8, 2014, 9:22PM    323.      0        0
## 3 Recommended        2      0           40 Sep 14, 2014, 8:47AM  1022.     79        2

Variables:

rating”: Recommended/Not recommended
nHelpful”: number voted helpful
nFunny”: number found funny
nScreenshots”: number of screenshots
date”: date posted
hours”: total game hours by the reviewer
nGames”: number of games
nReviews”: number of reviews

TBD: anyone knows steam and can help here?

I.8 Diamonds

It is a built-in dataset in ggplot2 library, so it is already loaded when you load the library. It contains price, shape, color and other information for 53940 diamonds. A sample of it looks

diamonds %>%
   sample_n(5)

## # A tibble: 5 × 10
##   carat cut       color clarity depth table price     x     y     z
##   <dbl> <ord>     <ord> <ord>   <dbl> <dbl> <int> <dbl> <dbl> <dbl>
## 1  0.31 Ideal     I     SI1      62.7    56   523  4.34  4.3   2.71
## 2  0.3  Very Good D     SI1      63      58   552  4.24  4.27  2.68
## 3  0.3  Ideal     E     VVS1     61.2    57   862  4.32  4.34  2.65
## 4  1.17 Very Good I     SI1      62.6    58  4826  6.64  6.71  4.18
## 5  1.72 Good      J     SI2      63.2    57  7741  7.56  7.6   4.79

Variables:

carat: mass of diamonds in caracts (ct), 1 ct = 0.2g
cut: cut describes the shape of diamond. There are five different cuts: Ideal is the best and Fair is the worst in these data. Better cuts make diamonds that are more brilliant.

Note: cut is an ordered factor, see Section 19.2.3.
color: There are 7 color levels, J (no color) is the best and D the worst, any color hue is considered not desirable.
clarity: measures the defects in diamonds, IF (internally flawless) is the best, and I1 is the worst.
depth, table: measures of the diamond shape
price: in $
x, y, z: diamond size, mm

I.9 Fatalities

In the book repo: fatalities.csv

The U.S. Traffic fatalities by state in 1980’s. This is a subset of dataset Fatalities in the AER package. Example:

read_delim("data/fatalities.csv") %>%
   sample_n(4)

## # A tibble: 4 × 4
##    year state fatal      pop
##   <dbl> <chr> <dbl>    <dbl>
## 1  1983 MN      555 4145008 
## 2  1988 MN      612 4307008 
## 3  1984 WA      746 4348992 
## 4  1987 MN      530 4246012.

Variables:

year: 1982-1988
state: only MN, OR, WA
fatal: total number of traffic fatalities
pop: population

I.10 Height-weight

In the book repo: height-weight.csv.

Synthetic dataset of five lines to demonstrate certain data properties. Here is the whole dataset, the meaning of columns is obvious:

read_delim("data/height-weight.csv")

## # A tibble: 5 × 4
##   sex      age height weight
##   <chr>  <dbl>  <dbl>  <dbl>
## 1 Female    16    173   58.5
## 2 Female    17    165   56.7
## 3 Male      17    170   61.2
## 4 Male      16    163   54.4
## 5 Male      18    170   63.5

I.11 Icecream

It is located in package Ecdat. It contains 30 four-weekly observations of ice cream consumption in 1950s in the U.S. Example:

data(Icecream, package = "Ecdat")  # 'Ecdat' must be installed
Icecream %>%
   sample_n(4)

##     cons income price temp
## 15 0.381     84 0.277   55
## 18 0.443     78 0.277   72
## 13 0.329     86 0.272   32
## 9  0.269     76 0.265   32

Variables:

cons: consumption of ice cream per head (in pints);
income: average family income per week (in US Dollars);
price: price of ice cream (per pint);
temp: average temperature (in Fahrenheit);

I.12 Ice extent

TBD: an explanatory figure of area/extent

In the book repo: ice-extent.csv.bz2.

National Snow & Ice Data Center (NSIDC) data about sea ice extent and area. Downloaded from U Colorado

A sample of data:

read_delim("data/ice-extent.csv.bz2") %>%
   sample_n(5)

## # A tibble: 5 × 7
##    year month `data-type` region extent  area  time
##   <dbl> <dbl> <chr>       <chr>   <dbl> <dbl> <dbl>
## 1  1994    10 Goddard     N        8.92  7.23 1995.
## 2  1982    12 Goddard     N       13.6  10.9  1983.
## 3  2021     3 Goddard     S        4.48  2.82 2021.
## 4  1989     1 Goddard     N       15.0  13.2  1989.
## 5  2010     8 Goddard     N        5.87  3.88 2011.

I haven’t found description of the variables, but these are fairly self-explanatory:

year
month: (1-12)
data-type: looks like the name of the satellite or another info provider
region: “N” for northern, “S” for southern hemisphere
extent: sea ice extent, in M km2. Extent is the sea surface area where the ice concentration is at least 15%.
area: sea ice surface area, M km2
time: a continuous time variable, made of year and month $\mathit{time} = \mathit{year} + \mathit{month}/12 - 1/24$. This describes roughly the middle of each month as measured in years.

I.13 Iris

Iris dataset is collected by Ronald Fisher 1936. It contains sepal and petal measures of 150 iris flowers of species setosa, versicolor and virginica (50 of each). It is an R built-in dataset and does not even have to be loaded, you can just use it as variable iris, a data frame with 150 rows and 5 columns.

The variables are

Sepal.Length: sepal length, in cm
Sepal.Width
Petal.Length
Petal.Width
Species: setosa/versicolor/virginica

A small example of it:

iris %>%
   sample_n(4)

##   Sepal.Length Sepal.Width Petal.Length Petal.Width   Species
## 1          5.8         4.0          1.2         0.2    setosa
## 2          6.3         2.5          5.0         1.9 virginica
## 3          4.9         3.1          1.5         0.2    setosa
## 4          7.2         3.6          6.1         2.5 virginica

I.14 Orange tree growth

It is an R built-in dataset, however, as that uses more complex data structures, a copy of it is in repo as a plain csv file: orange-trees.csv

Variables:

Tree: an ordered factor indicating the tree on which the measurement is made. The ordering is according to increasing maximum diameter.
age: a numeric vector giving the age of the tree (days since 1968/12/31)
circumference: a numeric vector of trunk circumferences (mm). This is probably “circumference at breast height”, a standard measurement in forestry.

I.15 Storms

In repo as storms.csv.

A tiny version of the storms dataset that is included in dplyr. It contains 6 storms only, and one line per the storm. Used to explain dplyr. Columns:

storm: storm name
wind: maximum wind speed (knots)
pressure: maximum air pressure (mbar). Note: this may be a different day than the date of the maximum wind speed.
date: date of the maximum wind speed (first date if multiple such dates).

Example:

## # A tibble: 3 × 4
##   storm   wind pressure date      
##   <chr>  <dbl>    <dbl> <date>    
## 1 Arlene    50     1015 1999-06-13
## 2 Ana       40     1013 1997-07-01
## 3 Arthur    45     1010 1996-06-21

I.16 Titanic

In repo as titanic.csv.bz2.

List of RMS Titanic passengers, their name, age and some more data, and whether they survived the shipwreck. It was collected by the investigation committee, and contains most of the passengers on the boat. The dataset is available in various sources, e.g. at kaggle. The variables are

pclass: Passenger Class (1 = 1st; 2 = 2nd; 3 = 3rd)
survived: Survival (0 = No; 1 = Yes)
name: Name
sex: Sex
age: Age
sibsp: Number of Siblings/Spouses Aboard
parch: Number of Parents/Children Aboard
ticket: Ticket Number
fare: Passenger Fare
cabin: Cabin
embarked: Port of Embarkation (C = Cherbourg; Q = Queenstown; S = Southampton)
boat: Lifeboat code (if survived)
body: Body number (if did not survive and body was recovered)
home.dest: The home/final destination of passenger

A small example of it:

pclass	survived	name	sex	age	parch	ticket	fare	cabin	embarked	boat	body	home.dest
3	1	Madsen, Mr. Fridtjof Arne	male	24	0	C 17369	7.1417	NA	S	13	NA	NA
2	1	Harper, Miss. Annie Jessie “Nina”	female	6	1	248727	33.0000	NA	S	11	NA	Denmark Hill, Surrey / Chicago
2	0	Parker, Mr. Clifford Richard	male	28	0	SC 14888	10.5000	NA	S	NA	NA	St Andrews, Guernsey
1	1	Greenfield, Mrs. Leo David (Blanche Strouse)	female	45	1	PC 17759	63.3583	D10 D12	C	7	NA	New York, NY

I.17 Tomato growth at different temperature

Synthetic data about three tomato plants, growing at different temperature. Used to demonstrate continuous and discrete values for plotting.

Columns:

plant: plant id, A, B, or C
age: in months
temperature: deg C
size: cm

Obviously, the units do not really matter for this toy dataset. Example:

read_delim("data/tomatoes-temperature.csv") %>%
   sample_n(4)

## # A tibble: 4 × 4
##   plant   age temperature  size
##   <chr> <dbl>       <dbl> <dbl>
## 1 C         1          25    60
## 2 A         3          15    80
## 3 A         1          15    40
## 4 A         2          15    60

I.18 Ukraine’s regional population

In repo as ukraine-oblasts-population.csv. Copied from the Wikipedia table 2024-03-03. Population as of 2015.

Example:

read_delim("data/ukraine-oblasts-population.csv") %>%
   head(3)

## # A tibble: 3 × 4
##   Prefecture            Population `Urban population` `Rural population`
##   <chr>                      <dbl>              <dbl>              <dbl>
## 1 Donetsk Oblast           4387702            3973317             414385
## 2 Dnipropetrovsk Oblast    3258705            2724872             533833
## 3 Kyiv                     2900920            2900920                 NA

The variables are self-explanatory.

I.19 Ukraine with regions

In repo as ukraine-with-regions_1530.geojson. The national borders and regional (oblast) borders of Ukraine in geojson format. Provided by Cartography Vectors.

The map:

library(sf)
library(ggplot2)
map <- read_sf("data/ukraine-with-regions_1530.geojson")
ggplot(map) +
   geom_sf()

National and regional (oblast) borders of Ukraine. Provided by Cartography Vectors.

I.20 US States

R has multiple small datasets about the US states. They are built-in variables, so you do not need to do anything special to load these. Examples:

## Full names of the states
state.name[1:5]

## [1] "Alabama"    "Alaska"     "Arizona"    "Arkansas"   "California"

## 2-letter abbriviations
state.abb[1:5]

## [1] "AL" "AK" "AZ" "AR" "CA"

Importantly, all these vectors contain data in the same order, so you can use names to find the value for the corresponding state.

I.21 Yin-yang

In repo as yinyang.csv.bz2.

An artificial dataset to demonstrate decision boundary for various classification methods.

Columns:

x, y: coordinates
c: 0/1, color

Example data:

## # A tibble: 3 × 3
##        x      y c    
##    <dbl>  <dbl> <fct>
## 1 -1.63   0.584 0    
## 2  0.624 -0.113 0    
## 3 -0.688  1.74  1

The scatterplot with c marked as color at right.