Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
223 changes: 223 additions & 0 deletions 02_activities/assignments/a2_survey_design_and_evaluation_frm.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,223 @@
# Assignment: Questionnaire Design and Sample Evaluation

## Requirements

The goal of this assignment is to practice developing and evaluating sampling materials.

### Part A - Survey Design:

Select one of the scenarios below and design a survey to meet the need(s) outlined in the prompt.

1. In two to three sentences, describe the purpose of your survey
2. Describe your target population, sampling frame, sampling units, and overall sampling strategy.
3. Write a 5-10 question survey to address your chosen scenario below.

##### Scenarios
1. You work in the Human Resources Department at a large tech company. Over the past few months, the company has been experiencing a high turnover rate across many of its departments, specifically within the entry- and lower-level positions. The company wishes to understand why this turnover is happening, and what changes need to occur to improve employee satisfaction.
2. You work for a Canadian national political party during a federal election. Throughout the campaign period, your party has seen relatively high approval ratings, but an opposing party is also polling favorably and may still have a chance to win the election. You are one month away from the election and you want to understand what voters want from your party and its leader in order to maintain your lead and eventually win the election.
3. You are a student researcher in the sociology department at the University of Toronto. You are working on a research project that concerns the relationship between music taste and age. This involves both comparisons between different people of different ages and comparisons of the same individual at different ages during their lifetime. You wish to understand to what extent age influences music taste, specifically as it relates to perceptions of popular music. Your results will be written into an academic paper that you hope to publish.

### Part B - Survey Evaluation:

For the **Canadian General Social Survey on Giving, Volunteering, and Participating, 2018 (cycle 33)**, conducted by Statistics Canada find any and all available documentation for the data gathered and identify and describe the survey features indicated below.

1. Sample type
2. Sample size
3. Target population
4. Sampling frame
5. Survey mode(s)
6. Timeline
7. Response rate
8. Weights
9. Data processing
10. Cleaning, imputation, etc
11. Sources of error
12. Limitations, known biases, etc
13. Link to documentation and any additional sources used


# Your Changes

## Part A - Survey Design:

The number of your chosen topic: #3

Describe the purpose of your survey:
To determine if there is a relationship/pattern in musical taste and age. Specifically, looking to see how older population percieves contemporary (pop) music across different musical eras. The survey is cross-sectional with emphasis on current music charts, and within age-group changes of each participant, to see if each person musical taste changes in a meaningful pattern (retrospective).


---

Describe your

- target population:Adults +18 recruited in GTA area
- sampling frame:participants will be recruited from the community via posting ads in libraries, unversities, elderly care centres. Participants are compensated via a $5 gift card of their choice from a variety of retailers.
- sampling units: a set number of participant (e.g. 100) per age groups defined by decade with the exception of 18-30 (12 years) and 70+.within each decade group researchers must ensure that the volume of female and male participants are similar with ± 15% difference.

- observational units:
Those who complete the survey. If funding and time to complete the study is sufficient this would be 500-600 participants.
---

Your 5-10 question survey:
```
1. write your question here...
How often do you listen to music?

Over an hour per day
less than an hour daily
Several times per week
a few times weekly
Several times per month
Infrequently (less than monthly)
Rarely or never
2. write your question here...
Which musical genres do you currently listen to most regularly? (Select all that applies)

Pop/Top 100
Rock/Alternative Rock
Hip-Hop/Rap
R&B/Soul
Country
Electronic/Dance
Jazz
Classical
Folk/Indie
Metal/Heavy Metal
Blues
punk
World/International (please specify which country)
not-listed here (please specify)

3. write your question here...

Please indicate your level of agreement with the following statements about contemporary popular music:
Scale: Strongly agree (5) / Agree (4) / Neither agree nor disagree (3) / Disagree (2) / Strongly disagree (1)
a) I generally enjoy popular music currently receiving significant airplay/streaming attention
b) Popular music today lacks the quality of music from earlier periods
c) I actively seek out information about emerging artists and new releases
d) Much of today's popular music sounds homogeneous
e) Contemporary popular music appears targeted toward younger demographic groups
f) Today's popular music demonstrates greater diversity than previous eras

4. write your question here...

Reflecting on the period when you were approximately 18-30 years old, which musical genres did you listen to most frequently during that time? (Select all that applies)

Pop/Top 100
Rock/Alternative Rock
Hip-Hop/Rap
R&B/Soul
Country
Electronic/Dance
Jazz
Classical
Folk/Indie
Metal/Heavy Metal
Blues
punk
World/International (please specify which country)
not-listed here (please specify)

5. write your question here...

Comparing your current musical preferences to those during ages 18-30, how would you characterize the degree of change in your taste?

a) Minimal change; preferences remain largely consistent
b) Modest evolution; core preferences persist with some variation
c) Moderate change; noticeable shifts in genre preferences
d) Substantial change; markedly different musical tastes
e) Not applicable (currently within this age range)

6. write your question here... (optional)

Current age: _____ years
Gender identity:
Man
Woman
Non-binary
Prefer to self-describe: ___________
Prefer not to answer


7. write your question here... (optional)
8. write your question here... (optional)
9. write your question here... (optional)
10. write your question here... (optional)
```

## Part B - Survey Evaluation:

Identify and describe survey features:
1. Sample Type:
Stratified probability sampling with dual-frame approach. Every unit had a known, non-zero selection probability—this is a probability sample where inference to the population is valid.

2. Sample Size:
n = 19,724 completed interviews, exceeding the target of ~15,000.

3. Target Population:
Non-institutionalized persons aged 15+ in private households across Canada's ten provinces. Exclusions: territories, institutions, reserves. The sampled population consists of actual respondents.

4. Sampling Frame:
Dual frame: (a) List frame from Address Register with linked telephone numbers, (b) RDD frame for households not in list frame. Stratified by province and CMA.

5. Survey Mode:
Mixed-mode: electronic questionnaire or computer-assisted telephone interviewing. Respondents selected their preferred language (English or French).

6. Timeline:
Data collection occurred from September 4, 2018, to December 28, 2018.

7. Response Rate:
41.9%

8. Weights:
Person-level weighting factor for estimating characteristics of the non-institutionalized population aged 15+. Adjustments include: multiplication factor for non-rejected non-volunteers due to rejective sampling; non-response adjustments (modeled using administrative data on income, household composition, etc.); calibration to match the 2017 Canadian Income Survey income distribution by province; alignment to independent age-sex-province population estimates for improved precision; bootstrap weights provided for design-based variance estimation.

9. Data Processing:
CATI electronic capture with built-in validation, coding of open-ended responses, automated/manual editing, derived variable construction.

10. Cleaning, Imputation:
Outlier detection for continuous variables (hours, donations), hot-deck imputation for missing key items, top-coding of extremes. "Don't Know/Refused" retained as categories.

11. Sources of Error:

Variability due to sampling; estimated using bootstrap method (high variability noted in publications via significance tests at 95% level). Non-sampling errors: Coverage errors (e.g., households without telephones or not on frame); non-response at household and individual levels; response errors; processing errors. Non-response and coverage biases are minimized through weighting, administrative modeling, and frame improvements, but some residual effects may remain.

12. Limitations, Known Biases, etc.:

Excludes the territories and full-time institutional residents. potential coverage bias from households without telephones. non-response bias reduced via modeling and adjustments but possible residuals. results may not be directly comparable to previous cycles due to the new electronic mode, updated volunteering definitions, and new content (e.g., online/social media participation questions). rejective sampling addresses over-representation of volunteers; linkage to tax records improves income quality but depends on consent and match rates; small biases possible from exclusions or frame limitations.

13. Link to Documentation:

Statistics Canada. (2020). *GSS Cycle 33 PUMF Documentation and User's Guide*. Catalogue no. 89M0026X.

IMDB: https://www23.statcan.gc.ca/imdb/p2SV.pl?Function=getSurvey&SDDS=4430





## Rubric

- All required components are present and complete **Complete / Incomplete**
- Choice of sampling strategy for Part A is justified and related to survey purpose **Complete / Incomplete**
- Information for Part B is complete and correct **Complete / Incomplete**

## Submission Information

🚨 **Please review our [Assignment Submission Guide](https://github.com/UofT-DSI/onboarding/blob/main/onboarding_documents/submissions.md)** 🚨 for detailed instructions on how to format, branch, and submit your work. Following these guidelines is crucial for your submissions to be evaluated correctly.

### Submission Parameters:
* Submission Due Date: `23:59 - 14 January 2026`
* The branch name for your repo should be: `assignment-2`
* What to submit for this assignment:
* This markdown file (a2_survey_design_and_evaluation.md) should be populated and should be the only change in your pull request.
* What the pull request link should look like for this assignment: `https://github.com/<your_github_username>/sampling/pull/<pr_id>`
* Open a private window in your browser. Copy and paste the link to your pull request into the address bar. Make sure you can see your pull request properly. This helps the technical facilitator and learning support staff review your submission easily.

Checklist:
- [ ] Create a branch called `assignment-2`.
- [ ] Ensure that the repository is public.
- [ ] Review [the PR description guidelines](https://github.com/UofT-DSI/onboarding/blob/main/onboarding_documents/submissions.md#guidelines-for-pull-request-descriptions) and adhere to them.
- [ ] Verify that the link is accessible in a private browser window.

If you encounter any difficulties or have questions, please don't hesitate to reach out to our team via the help channel in Slack. Our Technical Facilitators and Learning Support staff are here to help you navigate any challenges.