Sampling Distributions and the Central Limit Theorem
Statistics through experiments · 7 of 16 · CC BY-SA 4.0
Individual observations can be lopsided. Their averages have a surprising habit.
You have seen a sample mean move when you draw another sample. What if we kept every one of those estimates? They would form a distribution of their own.
The upper chart below describes individual values in a population. Start with one observation per sample and take 500 samples. With just one observation, each “average” is the observation itself. The two shapes should look similar, allowing for random variation.
Choose Keep these means. Then increase observations per sample to 30 and take 500 new samples. Predict which chart will change.
One sample becomes one mean
Dashed line: population mean = 20.3.
Read the chart as a table
| Value range | Count |
|---|---|
| 0.0–2.5 | 40 |
| 2.5–5.0 | 8 |
| 5.0–7.5 | 5 |
| 7.5–10.0 | 4 |
| 10.0–12.5 | 3 |
| 12.5–15.0 | 3 |
| 15.0–17.5 | 2 |
| 17.5–20.0 | 2 |
| 20.0–22.5 | 2 |
| 22.5–25.0 | 2 |
| 25.0–27.5 | 2 |
| 27.5–30.0 | 2 |
| 30.0–32.5 | 1 |
| 32.5–35.0 | 1 |
| 35.0–37.5 | 2 |
| 37.5–40.0 | 1 |
| 40.0–42.5 | 1 |
| 42.5–45.0 | 1 |
| 45.0–47.5 | 2 |
| 47.5–50.0 | 1 |
| 50.0–52.5 | 1 |
| 52.5–55.0 | 1 |
| 55.0–57.5 | 1 |
| 57.5–60.0 | 1 |
| 60.0–62.5 | 0 |
| 62.5–65.0 | 1 |
| 65.0–67.5 | 1 |
| 67.5–70.0 | 1 |
| 70.0–72.5 | 1 |
| 72.5–75.0 | 1 |
| 75.0–77.5 | 0 |
| 77.5–80.0 | 1 |
| 80.0–82.5 | 1 |
| 82.5–85.0 | 1 |
| 85.0–87.5 | 0 |
| 87.5–90.0 | 1 |
| 90.0–92.5 | 1 |
| 92.5–95.0 | 0 |
| 95.0–97.5 | 1 |
| 97.5–100.0 | 1 |
Each range includes its lower end and excludes its upper end, except the last range, which includes both.
Draw independently with replacement: each observation uses the same population again. Average the values, and place that one mean in the lower chart.
0 sample means recorded.
Dashed line: population mean = 20.3.
Read the chart as a table
| Value range | Count |
|---|---|
| 0.0–1.3 | 0 |
| 1.3–2.5 | 0 |
| 2.5–3.8 | 0 |
| 3.8–5.0 | 0 |
| 5.0–6.3 | 0 |
| 6.3–7.5 | 0 |
| 7.5–8.8 | 0 |
| 8.8–10.0 | 0 |
| 10.0–11.3 | 0 |
| 11.3–12.5 | 0 |
| 12.5–13.8 | 0 |
| 13.8–15.0 | 0 |
| 15.0–16.3 | 0 |
| 16.3–17.5 | 0 |
| 17.5–18.8 | 0 |
| 18.8–20.0 | 0 |
| 20.0–21.3 | 0 |
| 21.3–22.5 | 0 |
| 22.5–23.8 | 0 |
| 23.8–25.0 | 0 |
| 25.0–26.3 | 0 |
| 26.3–27.5 | 0 |
| 27.5–28.8 | 0 |
| 28.8–30.0 | 0 |
| 30.0–31.3 | 0 |
| 31.3–32.5 | 0 |
| 32.5–33.8 | 0 |
| 33.8–35.0 | 0 |
| 35.0–36.3 | 0 |
| 36.3–37.5 | 0 |
| 37.5–38.8 | 0 |
| 38.8–40.0 | 0 |
| 40.0–41.3 | 0 |
| 41.3–42.5 | 0 |
| 42.5–43.8 | 0 |
| 43.8–45.0 | 0 |
| 45.0–46.3 | 0 |
| 46.3–47.5 | 0 |
| 47.5–48.8 | 0 |
| 48.8–50.0 | 0 |
| 50.0–51.3 | 0 |
| 51.3–52.5 | 0 |
| 52.5–53.8 | 0 |
| 53.8–55.0 | 0 |
| 55.0–56.3 | 0 |
| 56.3–57.5 | 0 |
| 57.5–58.8 | 0 |
| 58.8–60.0 | 0 |
| 60.0–61.3 | 0 |
| 61.3–62.5 | 0 |
| 62.5–63.8 | 0 |
| 63.8–65.0 | 0 |
| 65.0–66.3 | 0 |
| 66.3–67.5 | 0 |
| 67.5–68.8 | 0 |
| 68.8–70.0 | 0 |
| 70.0–71.3 | 0 |
| 71.3–72.5 | 0 |
| 72.5–73.8 | 0 |
| 73.8–75.0 | 0 |
| 75.0–76.3 | 0 |
| 76.3–77.5 | 0 |
| 77.5–78.8 | 0 |
| 78.8–80.0 | 0 |
| 80.0–81.3 | 0 |
| 81.3–82.5 | 0 |
| 82.5–83.8 | 0 |
| 83.8–85.0 | 0 |
| 85.0–86.3 | 0 |
| 86.3–87.5 | 0 |
| 87.5–88.8 | 0 |
| 88.8–90.0 | 0 |
| 90.0–91.3 | 0 |
| 91.3–92.5 | 0 |
| 92.5–93.8 | 0 |
| 93.8–95.0 | 0 |
| 95.0–96.3 | 0 |
| 96.3–97.5 | 0 |
| 97.5–98.8 | 0 |
| 98.8–100.0 | 0 |
Each range includes its lower end and excludes its upper end, except the last range, which includes both.
All horizontal axes stay at 0–100. Each vertical axis counts its own observations. At most the latest 10,000 means are shown. Changing sample size clears the current means; changing population clears both runs.
Put a number on the narrowing
The population’s standard deviation is 27.17. For samples of 1, the standard error of the mean is 27.17 ÷ √1 = 27.17. The population’s spread stays the same; the estimates become steadier.
One dot stands for a whole sample
The lower chart is an observed picture of a sampling distribution: the distribution of a statistic across repeated samples drawn by the same rule. Each entry is a sample mean, not an individual measurement.
Try the other population shapes. An even spread, a long tail, and two separated peaks look quite different. Yet averages of many independent observations tend toward a common, rounded shape.
Increasing the number of samples makes that picture clearer. Increasing the number of observations per sample changes the distribution of the mean itself. These are the two different knobs you met with coins.
A moment to think
A name for steadier estimates
The standard deviation of a sampling distribution is called its standard error. It describes how much an estimate varies from sample to sample. The population’s standard deviation describes how much individuals vary.
For the independent draws here, multiplying sample size by four halves the standard error of the mean. Taking four times as many samples of the old size would not do that.
You have met a theorem
The Central Limit Theorem
For independent observations drawn from the same distribution with finite, positive variance, the distribution of their standardized mean approaches a standard normal distribution as sample size grows.
In ordinary units: for sufficiently large samples, the means are approximately bell-shaped around the population mean, with standard error equal to the population standard deviation divided by the square root of sample size.
SE(x̄) = σ / √n
σ, “sigma,” is the population standard deviation. n is observations per sample. SE measures the spread of sample means. This standard-error relationship is exact under these assumptions; the bell shape is an approximation.
What does “standardized” mean?
Subtract the population mean μ, then divide by the standard error: (x̄ − μ) / (σ / √n). This puts the estimate’s distance from its target in units of sampling uncertainty. That distribution approaches a normal distribution centered at zero with standard deviation one.
The population does not have to become normal. There is no universal “30 is enough” rule: strong skew or rare extremes can demand much larger samples. Dependence or infinite variance can break this version of the theorem.
The experiments illustrate the theorem; they do not prove it. See Penn State’s Central Limit Theorem lesson for a mathematical treatment.
Return to the experiment ↑A moment to think
Words you met
| Term | Meaning |
|---|---|
| Sampling variability | The change in an estimate across different samples. |
| Sampling distribution | The distribution of a statistic over repeated samples of the same size and design. |
| Standard error | The standard deviation of a sampling distribution. |
| Central Limit Theorem | Under its conditions, standardized sample means approach a standard normal distribution. |
Neighbors
- For the next mathematical steps, see Foundations for Inference and Inference for Means.
- Continue into the probability textbook’s Central Limit Theorem for the mathematics. Compare it with the Law of Large Numbers: settling toward a mean and approaching a bell shape are related, distinct ideas.
Written by June Kim. The conversational approach was inspired by Danielle Navarro’s Learning Statistics with R (CC BY-SA 4.0). The prose, experiments, and questions here were created for this book. This chapter’s text and illustrations are also shared under CC BY-SA 4.0.