HW 2 - Data wrangling

Homework
Important

This homework is due Wednesday, May 31st at 11:59pm.

The first step in the process of turning information into knowledge process is to summarize and describe the raw information - the data. In this assignment we explore data on college majors and earnings, specifically the data begin the FiveThirtyEight story “The Economic Guide To Picking A College Major”.

These data originally come from the American Community Survey (ACS) 2010-2012 Public Use Microdata Series. While this is outside the scope of this assignment, if you are curious about how raw data from the ACS were cleaned and prepared, see the code FiveThirtyEight authors used.

We should also note that there are many considerations that go into picking a major. Earnings potential and employment prospects are two of them, and they are important, but they don’t tell the whole story. Keep this in mind as you analyze the data.

Getting started

  • Go to the sta199-summer-1 organization on GitHub. Click on the repo with the prefix hw-2. It contains the starter documents you need to complete the homework assignment.

  • Clone the repo and start a new project in RStudio. See the Lab 0 instructions for details on cloning a repo and starting a new R project.

Workflow + formatting

Make sure to

  • Update author name on your document.
  • Label all code chunks informatively and concisely.
  • Follow the Tidyverse code style guidelines.
  • Make at least 3 commits.
  • Resize figures where needed, avoid tiny or huge plots.
  • Turn in an organized, well formatted document.

Packages

We’ll use the tidyverse package for much of the data wrangling and visualization and the scales package for better formatting of labels on visualizations.

Data

The data originally come from the fivethirtyeight package but we’ll use versions of the data that have been slightly modified to better suit this assignment. You can load the two datasets we’ll be using for this analysis with the following:

major_income_undergrad <- read_csv("data/major_income_undergrad.csv")
major_income_grad <- read_csv("data/major_income_grad.csv")

You can also take a quick peek at your data frames and view their dimensions with the glimpse function.

glimpse(major_income_undergrad)
Rows: 172
Columns: 12
$ major_code                            <dbl> 5601, 6004, 6211, 2201, 2001, 32…
$ major                                 <chr> "Construction Services", "Commer…
$ major_category                        <chr> "Industrial Arts & Consumer Serv…
$ undergrad_total                       <dbl> 86062, 461977, 179335, 37575, 53…
$ undergrad_employed                    <dbl> 73607, 347166, 145597, 29738, 43…
$ undergrad_employed_fulltime_yearround <dbl> 62435, 250596, 113579, 23249, 34…
$ undergrad_unemployed                  <dbl> 3928, 25484, 7409, 1661, 3389, 5…
$ undergrad_unemployment_rate           <dbl> 0.05066099, 0.06838588, 0.048422…
$ undergrad_p25th                       <dbl> 47000, 34000, 35000, 29000, 3600…
$ undergrad_median                      <dbl> 65000, 48000, 50000, 41600, 5200…
$ undergrad_p75th                       <dbl> 98000, 71000, 75000, 60000, 7800…
$ undergrad_sharewomen                  <dbl> 0.09071251, 0.69036529, 0.651659…
glimpse(major_income_grad)
Rows: 172
Columns: 11
$ major_code                       <dbl> 5601, 6004, 6211, 2201, 2001, 6206, 1…
$ major                            <chr> "Construction Services", "Commercial …
$ major_category                   <chr> "Industrial Arts & Consumer Services"…
$ grad_total                       <dbl> 9173, 53864, 24417, 5411, 9109, 19099…
$ grad_employed                    <dbl> 7098, 40492, 18368, 3590, 7512, 15157…
$ grad_employed_fulltime_yearround <dbl> 6511, 29553, 14784, 2701, 5622, 12304…
$ grad_unemployed                  <dbl> 681, 2482, 1465, 316, 466, 8324, 473,…
$ grad_unemployment_rate           <dbl> 0.08754339, 0.05775585, 0.07386679, 0…
$ grad_p25th                       <dbl> 110000, 89000, 100000, 85000, 83700, …
$ grad_median                      <dbl> 75000, 60000, 65000, 47000, 57000, 80…
$ grad_p75th                       <dbl> 53000, 40000, 45000, 24500, 40600, 50…

These two datasets have a trove of information. Three variables are common to both datasets:

  • major_code: Major code, FO1DP in ACS PUMS
  • major: Major description
  • major_category: Category of major from Carnevale et al

The remaining variables start with either grad_ or undergrad_ suffix, depending on which dataset they are in. The descriptions of these variables is as follows.

  • *_total: Total number of people with major
  • *_sample_size: Sample size (unweighted) of full-time, year-round ONLY (used for earnings)
  • *_employed: Number employed (ESR == 1 or 2)
  • *_employed_fulltime_yearround: Employed at least 50 weeks (WKW == 1) and at least 35 hours (WKHP >= 35)
  • *_unemployed: Number unemployed (ESR == 3)
  • *_unemployment_rate: Unemployed / (Unemployed + Employed)
  • *_p25th: 25th percentile of earnings
  • *_median: Median earnings of full-time, year-round workers
  • *_p75th: 75th percentile of earnings

Finally, undergrad_sharewomen is the proportion of women with the major, and we only have this information for undergraduates.

Let’s think about some questions we might want to answer with these data:

  • Which major has the lowest unemployment rate?
  • Which major has the highest percentage of women?
  • How do the distributions of median income compare across major categories?
  • How much are college graduates making?
  • How do incomes of those with a graduate degree compare to those with an undergraduate degree?

In the following exercises we aim to answer these questions.

Exercises

Exercise 1

Which majors have the lowest unemployment rate? Answer the question using a single data wrangling pipeline and focusing on undergraduates (major_income_undergrad). The output should be a tibble with the columns major, and unemployment_rate, with the major with the lowest unemployment rate on top, and displaying the majors with the lowest 5 unemployment rates. Include a sentence listing the majors and the unemployment rates (as percentages).

Exercise 2

Which majors have the highest percentage of women? Answer the question using a single data wrangling pipeline and focusing on undergraduates (major_income_undergrad). The output should be a tibble with the columns major, and undergrad_sharewomen, with the major with the highest proportion of women on top, and displaying the majors with the highest 5 proportions of women. Include a sentence listing the majors and the percentage of women with the major.

Render, commit (with a descriptive and concise commit message), and push. Make sure that you commit and push all changed documents and your Git pane is completely empty before proceeding.

Exercise 3

How much are college graduates making? For this exercise, focus on undergraduates (major_income_undergrad).

  1. Plot the distribution of all median incomes using a histogram with an appropriate binwidth.

  2. Calculate the mean and median for median income. Based on the shape of the histogram, determine which of these summary statistics is useful for describing the distribution.

  3. Describe the distribution of median incomes of college graduates across various majors based on your histogram from part (a) and incorporating the statistic you chose in part (b) to help your narrative. Hint: Mention shape, center, spread, any unusual observations.

Now is a good time to render, commit (with a descriptive and concise commit message), and push again. Make sure that you commit and push all changed documents and your Git pane is completely empty before proceeding.

Exercise 4

How do the distributions of median income compare across major categories? For this exercise, focus on undergraduates (major_income_undergrad).

  1. Calculate a the minimum, median, and maximum median income per major category as well as the number of majors in each category. Your summary statistics should be in decreasing order of median median income.

  2. Create box plots of the distribution of median income by major category.

    • The variable major_category should be on the y-axis and undergrad_median on the x-axis.

    • The order of the boxes in your plot should match the order in your summary table from part (a).

    • Use color to enhance your plot, and turn off any legends providing redundant information.

Wrap up

Submission

  • Go to http://www.gradescope.com and click Log in in the top right corner.
  • Click School Credentials Duke Net ID and log in using your Net ID credentials.
  • Click on your STA 199 course.
  • Click on the assignment, and you’ll be prompted to submit it.
  • Mark all the pages associated with exercise. All the pages of your homework should be associated with at least one question (i.e., should be “checked”). If you do not do this, you will be subject to lose points on the assignment.
  • Select the first page of your PDF submission to be associated with the “Workflow & formatting” question.

Grading

  • Exercise 1: 10 points
  • Exercise 2: 10 points
  • Exercise 3: 10 points
  • Exercise 4: 15 points
  • Workflow + formatting: 5 points
  • Total: 50 points