Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 28 additions & 19 deletions 02_activities/assignments/a2_survey_design_and_evaluation.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@ Select one of the scenarios below and design a survey to meet the need(s) outlin
1. In two to three sentences, describe the purpose of your survey
2. Describe your target population, sampling frame, sampling units, and overall sampling strategy.
3. Write a 5-10 question survey to address your chosen scenario below.
Q1 - What is your age? (Enter a whole number)
Q2 - On the Sliding Scale of 1 to 7, 7 being everyday, how often do you listen to music?
Q3 -

##### Scenarios
1. You work in the Human Resources Department at a large tech company. Over the past few months, the company has been experiencing a high turnover rate across many of its departments, specifically within the entry- and lower-level positions. The company wishes to understand why this turnover is happening, and what changes need to occur to improve employee satisfaction.
Expand Down Expand Up @@ -40,39 +43,45 @@ For the **Canadian General Social Survey on Giving, Volunteering, and Participat

## Part A - Survey Design:

The number of your chosen topic: `#`
The number of your chosen topic: #3

Describe the purpose of your survey:
```
write your answer here...
```
To determine whether there is an association between music genre preference and age of Adult Canadians. Furthermore, to examine if there are moderating or mediating factors that can help explain this correlation, such as listening habits and musical training.

Describe your target population, sampling frame, sampling units, and observational units:
```
write your answer here...
```
The target population are all Adult Canadians (18+)- it is necessary assumption here that every adult Canadian at some point in their life has listened to music. The sampling frame is mean't to closely approximate the population, so perhaps a list of every individual in all the neighbourhoods across Canada - such as a YellowPages Phone Book or even accessing cencus data. The sampling units themselves are the individual Canadian Adults, anyone who is 18+ and known to the Canadian Government. My overall sampling strategy would likely use stratified sample, so splitting regions of Canada or even neighbourhoods into strata and sampling from each one. The inclusion criteria is anyone who is a Canadian Citizen/PR and 18+ who has listened to music in their life. The exclusion criteria would be anyone who is a minor or unregistered/undocumented Canadian or immigrant status. I will recruit using multiple methods including at local community townhalls/institutions, civic advertisements, mailing lists.

Your 5-10 question survey:
```
1. write your question here...
2. write your question here...
3. write your question here...
4. write your question here...
5. write your question here...
6. write your question here... (optional)
7. write your question here... (optional)
8. write your question here... (optional)
1. What is your age? (Enter a whole number)
2. Select your province/Territory of residency. (Have a drop down menu with all 13 options)
3. On the Sliding Scale of 1 to 7, 7 being everyday, how often do you listen to music?
4. Do you prefer music in a major, minor, or mix of major-minor key? (force choice)
5. Do you prefer music that is happy or sad? (force choice)
6. On a Sliding Scale of 0 to 100, how fast do you like your music?
7. Do you tend to listen to music in the morning, afternoon, or evenings?
8. What is your musical training level? (Musical Training Answers Options: Never taken lessons, taken lessons, completed diploma, University music graduate)
9. write your question here... (optional)
10. write your question here... (optional)
10. write your question here...
```

## Part B - Survey Evaluation:

Identify and describe survey features:

```
write your answer here
```
1. Sample type - The 5.2 documentation on page 14 of the g uide shows that they used stratification, where the 10 provinces were divded into geographical regions for a strata 27. The document 3.1 shows us that they also used mixed-methods approach when it came to targeting the individual household, for example in selecting one specific individual per household and also used age - order selection. 5.5 also mentions that they used rejective sampling
2. Sample size - target was 20,000, but n = 16,149
3. Target population - according to 5.1, all people 15+ in Canada, except for those who are residents in institutions or residents of the territories
4. Sampling frame - frame was established according to 5.3 using stats canada list of phone numbers and the Address Register (all dwellings in 10 provinces)
5. Survey mode(s) - 3.1 indicates two types, people were either contact via letter and than asked to complete the electronic questionnaire, or people were called via telephone and interviewed
6. Timeline - data collection was from september to december of 2018
7. Response rate - 41.9% according to section 6
8. Weights - they note at the beginning of section 8 that "the principle behind estimation is that each person selected in the sample represents (in addition to himself or herself) several other persons not in the sample" meaning that each person was weighted as several people. They had multiple weights: they adjusted weights to also "represent non-responding telephone numbers," adjusted weights for the fact that some households had higher probability of being selected, stratum adjustment, income adjustment , age/sex adjustment etc.
9. Data processing - In section 4, they said "most of the ongoing data processing steps are standard, including consistency edits, family edits and imputation" They used a data capture program that automated and simplified much of the data processing. For free-response answers, they coded (according to 7.2) the repsonses according to certain categories or themes
10. Cleaning, imputation, etc - In 7.3, they discuss how they removed out-of-scope respondants in the data processing/cleanup stages. They also eliminated duplicate records and needed to perform verification of identity. In a small number of cases, recordings missing/with wrong information were "correct deterministically or imputed." They said that their strategy behind imputing was to maintain internal coherent underlying data structure. They also used donor imputation to fill missing data for several aspects such as partial non-response.
11. Sources of error - In section 9, they discuss sampling variability and potential errors.Mostly these are related to non-sampling errors or non-response of the survey results. Personal income had a high imputation rate of 18.1%. 9.2.2. also acknowledges standard sampling error that we have discussed in class.
12. Limitations, known biases, etc - one limitation that i identify is that they did not survey the territories, which is a chunk of the Canadian population! They also only surveyed households, and not every individual resides in a household!
13. Link to documentation and any additional sources used: https://utoronto-my.sharepoint.com/:b:/g/personal/evankp_chan_mail_utoronto_ca/IQDxu40YnLr_QpbvU6OYNoN2ASBEkJqUKxdGqG1Eh-F-1wM?e=AlsnsM

## Rubric

Expand Down