Sampling Distributions

Study Sheet

Sampling Distributions

How a statistic varies from sample to sample

The Sampling Distribution of a Mean

Concept
Means of samples

Take a random sample of size nn from a population with mean μ\mu and standard deviation σ\sigma. The sample mean xˉ\bar x is itself random: its mean is μ\mu and its standard deviation is σn\dfrac{\sigma}{\sqrt n}. By the central limit theorem, for large nn (about 3030 or more) xˉ\bar x is approximately normal whatever the population looks like.

The n\sqrt n is the most consequential square root in statistics. Averaging four measurements halves the noise; averaging a hundred cuts it by ten. To halve the error again you must quadruple the sample — which is why precision is expensive and why polls of a thousand people are about as good as it gets for the price.

Example
Probability for a mean
one bottlemean of 25470480490496500510520530
Dashed: single bottles (SD 10). Solid: means of 25 (SD 2). The tail below 496 is tiny for means.

Bottles hold mean 500500 mL, SD 1010 mL, and the fill is roughly normal. For a sample of 2525 bottles, xˉ\bar x has mean 500500 and SD 10/25=210/\sqrt{25} = 2. P(xˉ<496)=P(Z<496−5002)=P(Z<−2)≈0.023P(\bar x < 496) = P\left(Z < \tfrac{496 - 500}{2}\right) = P(Z < -2) \approx 0.023. A single bottle below 496496 is common (z=−0.4z = -0.4); a whole sample averaging that low is rare.

Example
A skewed population

Waiting times at a counter are strongly right-skewed with mean 44 minutes and SD 44 minutes. For 6464 customers, the CLT still applies: xˉ\bar x is approximately normal with mean 44 and SD 4/64=0.54/\sqrt{64} = 0.5. So P(xˉ>5)≈P(Z>2)≈0.023P(\bar x > 5) \approx P(Z > 2) \approx 0.023, even though individual waits are nothing like normal.

Side note
Where it goes wrong

Using σ\sigma instead of σ/n\sigma/\sqrt n for a sample mean. For the bottles, z=(496−500)/10=−0.4z = (496 - 500)/10 = -0.4 answers a question about ONE bottle, not about the average of 2525.

Try it
Try it: sample size

Population SD is 1212. How large a sample makes the SD of xˉ\bar x equal to 22?

Show the answer

12/n=212/\sqrt n = 2, so n=6\sqrt n = 6 and n=36n = \mathbf{36}.

The Sampling Distribution of a Proportion

Concept
Proportions

A sample proportion p^\hat p of successes has mean pp and standard deviation p(1−p)n\sqrt{\dfrac{p(1-p)}{n}}, approximately normal when np≥10np \ge 10 and n(1−p)≥10n(1-p) \ge 10. It is the binomial count divided by nn.

Dividing a binomial count by nn divides its mean npnp by nn and its SD np(1−p)\sqrt{np(1-p)} by nn, which gives exactly these formulas. The spread depends on the sample size, not on the size of the population (as long as the sample is a small fraction of it), which is why a well-drawn sample of 10001000 describes a city and a country equally well.

Example
A poll
.445.470.495.520.545.570.595poll 0.47
Sample proportions from n = 400 when p = 0.52: a result of 0.47 or lower sits in the far left tail.

If 52%52\% of voters support a measure and n=400n = 400: the SD of p^\hat p is 0.52⋅0.48/400≈0.025\sqrt{0.52\cdot0.48/400} \approx 0.025. A poll showing 47%47\% would be (0.47−0.52)/0.025=−2(0.47 - 0.52)/0.025 = -2 SDs from the truth — possible, but it would happen only about 2%2\% of the time.

Example
Quadrupling the sample

The same poll with n=1600n = 1600: SD =0.52⋅0.48/1600≈0.0125= \sqrt{0.52\cdot0.48/1600} \approx 0.0125, exactly half of 0.0250.025. Four times the work buys twice the precision.

Side note
Where it goes wrong

Applying the normal approximation when np<10np < 10. For a trait with p=0.03p = 0.03 and n=100n = 100, np=3np = 3: the distribution of p^\hat p is squashed against 00 and skewed, and normal-curve probabilities for it are unreliable.

Try it
Try it: proportion

True proportion p=0.3p = 0.3, sample size n=210n = 210. Find the SD of p^\hat p.

Show the answer

0.3⋅0.7/210=0.21/210=0.001≈0.0316\sqrt{0.3\cdot0.7/210} = \sqrt{0.21/210} = \sqrt{0.001} \approx \mathbf{0.0316}. (Conditions: np=63np = 63 and n(1−p)=147n(1-p) = 147, both at least 1010.)

The Central Limit Theorem

Concept
Averages become normal

Whatever the shape of the population, the distribution of the sample mean xˉ\bar x of nn independent observations gets closer to a normal curve as nn grows, with mean μ\mu and SD σ/n\sigma/\sqrt n. The more skewed the population, the larger the nn needed; for moderate skew, n≈30n \approx 30 is usually enough.

Averaging combines many independent pieces, and the extremes need ALL the pieces to be extreme at once, which becomes very unlikely: a mean of 11 from four dice needs four 11s (11 chance in 12961296), while there are many ways to average near 3.53.5. So probability piles up in the middle and thins out symmetrically in the tails. The theorem is about the MEAN; the population itself does not change shape.

Example
Two dice
13.561 die13.56mean of 213.56mean of 4same centre 3.5; narrower and more bell-shaped as n grows
Exact distributions of the mean of 1, 2 and 4 dice: from flat, to triangle, to bell.

One die is flat: each value has probability 1/61/6. The mean of two dice takes values 1,1.5,…,61, 1.5, \ldots, 6 with a triangular distribution: xˉ=3.5\bar x = 3.5 happens for the 66 pairs summing to 77, probability 6/36=1/66/36 = 1/6, while xˉ=1\bar x = 1 needs a double one, 1/361/36. Already the middle is favoured.

Example
Four dice

A single die has σ=35/12≈1.708\sigma = \sqrt{35/12} \approx 1.708. The mean of 44 dice has SD 1.708/4≈0.8541.708/\sqrt4 \approx 0.854 and a clearly bell-shaped distribution, even though the population (one die) is flat. The centre never moves: it is 3.53.5 at every nn.

Side note
Where it goes wrong

Thinking the data become normal. Rolling a die 10001000 times gives a flat histogram of 10001000 values — the population is still uniform. What is normal is the distribution of the AVERAGE across many repetitions of the sampling.

Try it
Try it: nine dice

What is the standard deviation of the mean of 99 fair dice?

Show the answer

σ/n=1.708/9=1.708/3≈0.569\sigma/\sqrt n = 1.708/\sqrt9 = 1.708/3 \approx \mathbf{0.569}.

Why It Works

The sample mean is xˉ=X1+⋯+Xnn\bar x = \dfrac{X_1 + \cdots + X_n}{n}. Variances of independent terms add: Var(X1+⋯+Xn)=nσ2\text{Var}(X_1 + \cdots + X_n) = n\sigma^2. Dividing a variable by nn divides its variance by n2n^2: Var(xˉ)=nσ2n2=σ2n\text{Var}(\bar x) = \dfrac{n\sigma^2}{n^2} = \dfrac{\sigma^2}{n}. Taking the square root gives σ/n\sigma/\sqrt n. The central limit theorem is a deeper fact — the SHAPE becomes normal — but the size of the spread is just this arithmetic.