Data doesn’t just happen.
Plan ahead for good data.
Planning is easy.
1. The Sample | Chapter 8 |
2.Sample Size | Chapter 10 |
3.Recruitment Method | Chapter 10 |
4.Deployment | Chapter 10 |
The Sample
You cannot study the whole population: you study a sample.
Your sample is an uncertain guide to the population.
You manage that uncertainty by carefully design:
- Sample size
- Recruitment method
- Participant deployment
Sample Size
How many participants are you going to need?
The smaller the effect size, the more participants you need.
You can use power analysis to calculate the ideal sample size.
Here is a very rough guide.
- small effect (r<0.15): at least 150 participants
- medium effect (r<0.3): at least 75 participants
- large effect (r>0.5): 50 participants
Often these numbers are unrealistic and you must live with what you can get.
Recruitment method
The way you recruit your participants matters. Here are the common ones, from best (top) to worst (bottom):
- Random: everyone is recruited by pure chance
This is unrealistic as usually not everyone has the same chance of beoing recruited - Stratified: we try and match the characteristics of the population (eg. keeping the gender balance)
- Opportunity: whoever we can find.
More or less what we have to do. We may miss out on whole groups of participants who might matter. - Word of mouth (and social media): we pass invitations out through our friends and other contacts.
Not recommended as this leads to potential issues with participant non-independence (they are all too similar).
The deployment of our participants
Sometimes we can choose between two methods of using our participants. If the IV is Categorical and refers to different situations (eg test phases or experimental conditions), then we can choose between:
- Between participants: each participant belongs to only one group or condition
- Within participants: each participant belongs successively to each group or condition
Within designs are more powerful (not least because we get more data from each participant). But they have potential problems, especially order effects.
The Plan (part 2)
The research plan can be finalized. We can also capture these new decisions in a simple table:
Design | Decision |
---|---|
Sample size | 150 |
Recruitment | Opportunity |
Deployed as | Between |