Random Variables

Study Sheet

Random Variables

Expected value, variance and combining variables

Discrete Random Variables

Concept
Expected value and variance

A random variable XX assigns a number to each outcome; its probability distribution lists each value with its probability (summing to 11). E[X]=μ=∑x p(x)E[X] = \mu = \sum x\,p(x) is the long-run average; Var(X)=∑(x−μ)2p(x)\text{Var}(X) = \sum(x - \mu)^2p(x) and σ=Var(X)\sigma = \sqrt{\text{Var}(X)}. A shortcut: Var(X)=E[X2]−μ2\text{Var}(X) = E[X^2] - \mu^2.

Expected value is a weighted mean with probabilities as weights, so everything you know about means transfers: it is linear, it can be a value XX never takes, and it is what the average of many repetitions settles toward. If the distribution were drawn as a bar chart cut from cardboard, μ\mu is where it would balance. Variance measures how far from that balance point a single trial typically lands.

Example
A raffle

A $2\$2 ticket wins $100\$100 with probability 0.010.01 and nothing otherwise. Net winnings XX: 9898 with probability 0.010.01, −2-2 with probability 0.990.99. E[X]=98(0.01)+(−2)(0.99)=0.98−1.98=−1E[X] = 98(0.01) + (-2)(0.99) = 0.98 - 1.98 = -1: over many tickets you lose a dollar per ticket on average, though no single ticket ever loses exactly a dollar.

Example
Cars per household
00.110.420.330.2μ = 1.6
The distribution balances at μ = 1.6, a value no household actually has.

In a town, the number of cars XX per household is 0,1,2,30, 1, 2, 3 with probabilities 0.1,0.4,0.3,0.20.1, 0.4, 0.3, 0.2 (these sum to 11, as a distribution must). E[X]=0+0.4+0.6+0.6=1.6E[X] = 0 + 0.4 + 0.6 + 0.6 = 1.6. E[X2]=0+0.4+1.2+1.8=3.4E[X^2] = 0 + 0.4 + 1.2 + 1.8 = 3.4, so Var(X)=3.4−1.62=0.84\text{Var}(X) = 3.4 - 1.6^2 = 0.84 and σ≈0.92\sigma \approx 0.92.

Side note
Where it goes wrong

Reading the expected value as the most likely value. A fair die has E[X]=3.5E[X] = 3.5, a value it can never show, and in the cars example the most likely count is 11, not 1.61.6. Expected value is a long-run average, not a prediction for one trial.

Try it
Try it: expected value

Roll a fair die; you win the number shown in dollars. Find E[X]E[X] and Var(X)\text{Var}(X).

Show the answer

E[X]=(1+2+⋯+6)/6=3.5E[X] = (1 + 2 + \cdots + 6)/6 = 3.5. E[X2]=(1+4+9+16+25+36)/6=91/6E[X^2] = (1 + 4 + 9 + 16 + 25 + 36)/6 = 91/6, so Var=91/6−12.25=35/12≈2.92\text{Var} = 91/6 - 12.25 = \mathbf{35/12} \approx 2.92.

Expected Value in Decisions

Concept
Fair, favourable, unfavourable

A game is fair when the expected net gain is 00. Expected value prices risk: an insurer charges more than the expected payout, a casino offers games with negative expected value for the player. Linearity, E[X+Y]=E[X]+E[Y]E[X + Y] = E[X] + E[Y], holds for ANY random variables, independent or not, which makes expectations of complicated counts easy.

The law of large numbers is what gives expected value its authority: over many independent plays the average gain gets close to E[X]E[X], so a negative expectation is a guaranteed loss in the long run however lucky any single night is. Linearity is the trick that makes hard problems easy: write a complicated count as a sum of simple 0/1 indicators, and add up their probabilities.

Example
A dice game
2132435465768594103112121ways (of 36)
Sums of two dice: 7 is the most likely, with 6 of 36 outcomes.

It costs $1\$1 to play; roll two dice and if the sum is 77 you are paid $5\$5. Six of the 3636 outcomes sum to 77, so P(7)=1/6P(7) = 1/6. Net gain: +4+4 with probability 1/61/6, −1-1 with probability 5/65/6. E=4/6−5/6=−1/6≈−$0.17E = 4/6 - 5/6 = -1/6 \approx -\$0.17 per play. A fair payout would be $6\$6.

Example
Insurance

A policy costs $50\$50 a year and pays $20,000\$20{,}000 if a loss occurring with probability 1/5001/500 happens. The insurer's expected payout per policy is 20000/500=$4020000/500 = \$40, so it expects to keep $10\$10 per policy. The customer accepts a small expected loss to avoid a small chance of a ruinous one — expected value is not the only thing that matters to a person, but it is what matters to a company selling a million policies.

Example
Linearity: letters in envelopes

nn letters are put into nn addressed envelopes at random. Expected number in the right envelope? Let Xi=1X_i = 1 if letter ii is right, else 00; P(Xi=1)=1/nP(X_i = 1) = 1/n. Then E[X]=E[X1]+⋯+E[Xn]=n⋅1n=1E[X] = E[X_1] + \cdots + E[X_n] = n\cdot\tfrac1n = 1, for every nn. The XiX_i are not independent, and it does not matter: linearity needs no independence.

Side note
Where it goes wrong

Forgetting the cost. In the dice game, "win $5\$5 with probability 1/61/6, so E=5/6≈$0.83E = 5/6 \approx \$0.83" ignores the dollar paid to play; the net outcomes are +4+4 and −1-1. Define the random variable as the NET gain before computing anything.

Try it
Try it: guessing penalty

A multiple-choice question has 55 options. A right answer scores +1+1, a wrong one −14-\tfrac14. What is the expected score of a random guess?

Show the answer

E=15(1)+45(−14)=15−15=0E = \tfrac15(1) + \tfrac45\left(-\tfrac14\right) = \tfrac15 - \tfrac15 = \mathbf{0}. The penalty is designed to make blind guessing exactly fair.

Combining Random Variables

Concept
Linear combinations

For any random variables, E[aX+bY]=aE[X]+bE[Y]E[aX + bY] = aE[X] + bE[Y]. For independent XX and YY, Var(aX+bY)=a2Var(X)+b2Var(Y)\text{Var}(aX + bY) = a^2\text{Var}(X) + b^2\text{Var}(Y) — variances add even when you subtract, because b2b^2 is positive. For one variable, E[a+bX]=a+bE[X]E[a + bX] = a + bE[X] and σa+bX=∣b∣σX\sigma_{a + bX} = |b|\sigma_X.

Standard deviations never add; variances do, and only for independent variables. That makes standard deviations combine like the sides of a right triangle: σX±Y=σX2+σY2\sigma_{X \pm Y} = \sqrt{\sigma_X^2 + \sigma_Y^2}. The difference of two independent measurements is MORE variable than either one, not less, which is why comparing two groups needs bigger samples than measuring one.

Example
Total weight

Boxes weigh 1010 kg on average with SD 11 kg, independently. Five boxes: mean 5⋅10=505\cdot10 = 50 kg; variance 5⋅12=55\cdot1^2 = 5; SD 5≈2.24\sqrt5 \approx 2.24 kg — not 55 kg. Random errors partly cancel: some boxes are heavy, some light.

Example
Two exam sections
SD 8SD 6SD 10
Independent spreads add as squares: √(6² + 8²) = 10, for a sum or a difference.

Math scores have SD 66 and verbal scores SD 88, independent. The total has SD 36+64=10\sqrt{36 + 64} = 10, and so does the difference math −- verbal: Var(M−V)=Var(M)+(−1)2Var(V)=100\text{Var}(M - V) = \text{Var}(M) + (-1)^2\text{Var}(V) = 100. The spreads combine like the legs of a right triangle.

Example
A linear cost

Using the cars-per-household variable (μ=1.6\mu = 1.6, σ≈0.917\sigma \approx 0.917), a parking permit costs $20\$20 plus $5\$5 per car: C=20+5XC = 20 + 5X. Then E[C]=20+5⋅1.6=$28E[C] = 20 + 5\cdot1.6 = \$28 and σC=5⋅0.917≈$4.58\sigma_C = 5\cdot0.917 \approx \$4.58. The fixed $20\$20 moves the centre and adds no spread.

Side note
Where it goes wrong

Writing σX−Y=σX−σY\sigma_{X - Y} = \sigma_X - \sigma_Y. It is σX2+σY2\sqrt{\sigma_X^2 + \sigma_Y^2} for independent variables. With equal SDs the wrong formula would give a difference with no variability at all — two random things whose difference is certain.

Try it
Try it: a difference

XX has SD 33 and YY has SD 44, independent. Find the SD of X−YX - Y.

Show the answer

Variances add: Var(X−Y)=9+16=25\text{Var}(X - Y) = 9 + 16 = 25, so the SD is 25=5\sqrt{25} = \mathbf{5}.

Why It Works

For independent X,YX, Y with means μX,μY\mu_X, \mu_Y: Var(X+Y)=E[(X−μX+Y−μY)2]=Var(X)+Var(Y)+2E[(X−μX)(Y−μY)]\text{Var}(X + Y) = E[(X - \mu_X + Y - \mu_Y)^2] = \text{Var}(X) + \text{Var}(Y) + 2E[(X - \mu_X)(Y - \mu_Y)]. Independence makes the cross term the product of two expectations, each zero. Without independence the cross term is 2 Cov(X,Y)2\,\text{Cov}(X, Y), which is where correlation enters — and why the formula fails for, say, a value and itself: Var(X+X)=4Var(X)\text{Var}(X + X) = 4\text{Var}(X), not 2Var(X)2\text{Var}(X).