It is often desirable to use empirical data to compare two or more hypotheses, usually to identify which hypothesis should be preferred.
This is relatively straightforward. We make a world in which the hypotheses we wish to compare all exist. Then we ask, in that world, which hypothesis is the more likely to have produced the data we have.
An example world
Imagine that we have two hypotheses:
We build a world that has populations corresponding to each hypothesis. Here are the sampling distributions for the two hypotheses and (at the back), for the whole world.
Each of these sampling distributions shows the distribution of sample values for that would be found by sampling repeatedly from that population.
The sample
Now suppose that we have a sample:
We can easily ask how often such a sample might arise from each possible population. The next figure shows the answer. Not surprisingly, the population that is preferred is .
The issue
In this world, the first one occurs 10 times as often as the second.