Conditional Probability, Independence & Bayes

A fair die is rolled. Given that the result is even, what is the probability it is greater than $3$?

Solution. Let $B=\{2,4,6\}$ and $A=\{4,5,6\}$. Then $A\cap B=\{4,6\}$, so

Knowing “even” throws away the odd outcomes; we only compare within $\{2,4,6\}$.

Two cards are drawn from a standard $52$-card deck without replacement. What is the probability the second is a heart given the first was a heart?

Solution. After removing one heart, $12$ hearts remain among $51$ cards, so

Conditioning literally updates the deck.

Two cards are drawn without replacement. Probability both are aces?

Solution. Multiply along the path:

A box has $2$ red and $3$ blue balls. Draw two without replacement. Probability of exactly one red?

Solution. Two paths give “one red”: RB and BR.

Multiply along each path, then add the paths that satisfy the event.

Draw a card, replace it, reshuffle, draw again. Are “first is a heart” and “second is a heart” independent?

Solution. With replacement the deck resets: $P(\text{2nd heart}\mid\text{1st heart}) = \tfrac14 = P(\text{2nd heart})$, so yes. Without replacement it was $\tfrac{12}{51}\neq\tfrac14$, so no. Replacement is the usual switch between independent and dependent.

Roll two fair dice. Let $A=$ “first die is $\le 2$” and $B=$ “sum is $7$”. Independent?

Solution. $P(A)=\tfrac{2}{6}=\tfrac13$, $P(B)=\tfrac{6}{36}=\tfrac16$. The pairs with first die $\le 2$ and sum $7$ are $(1,6),(2,5)$, so $P(A\cap B)=\tfrac{2}{36}=\tfrac{1}{18}$. Check: $P(A)P(B)=\tfrac13\cdot\tfrac16=\tfrac{1}{18}$. Equal, so independent.

Urn 1 has $3$ red, $1$ white; Urn 2 has $1$ red, $3$ white. Flip a fair coin to pick an urn, then draw a ball. Probability the ball is red?

Solution. Partition by the urn:

Machines A, B, C make $50\%, 30\%, 20\%$ of output with defect rates $1\%, 2\%, 3\%$. Probability a random item is defective?

Solution.

A weighted average of the defect rates, weighted by production share.

Using the two-urn setup above, a red ball was drawn. Probability it came from Urn 1?

Solution.

The red ball is evidence that shifts belief toward the red-heavy urn.

For the three machines above, a defective item is found. Probability it was machine C's?

Solution.

Even though C makes only $20\%$ of output, its high defect rate makes it responsible for over a third of defects.

Three doors hide one car and two goats. You pick a door; the host --- who knows where the car is --- opens a different door revealing a goat, then offers a switch. Should you switch?

Solution. Your first pick is right with probability $\tfrac13$ and wrong with probability $\tfrac23$. The host's guaranteed goat-reveal gives no new information about your original door, so:

• 2pt
• If you initially picked the car ($\tfrac13$), switching loses.
• If you initially picked a goat ($\tfrac23$), the host is forced to reveal the other goat, so the remaining door hides the car --- switching wins.

Therefore $P(\text{win by switching}) = \tfrac23$, double the $\tfrac13$ from staying. The key is that the host's choice is constrained by knowledge; a host opening a random door would change the analysis.

A disease affects $1$ in $1000$ people. A test is $99\%$ accurate both ways (sensitivity $99\%$, specificity $99\%$). You test positive. What is the probability you actually have the disease?

Solution. Let $D$ be disease, $+$ a positive test. Priors $P(D)=0.001$, $P(D^c)=0.999$.

By Bayes,

Only about $9\%$! Because the disease is rare, the many false positives from the huge healthy population swamp the few true positives. The base rate dominates.

A family has two children.

• 2pt
• “At least one is a boy.” Probability both are boys? Sample space $\{BB,BG,GB,GG\}$ is equally likely; condition on at least one boy: $\{BB,BG,GB\}$, of which $BB$ is one. Answer $\tfrac13$.
• “The older is a boy.” Probability both boys? Now condition on $\{BB,BG\}$: answer $\tfrac12$.

Same family, different information, different answers --- see Going Deeper for why the phrasing matters.

The rare disease ($P(D)=0.001$) test above comes back positive; a second independent test (same accuracy) is also positive. Now find $P(D\mid ++)$.

Solution. Use the first posterior $P(D\mid +)\approx 0.0902$ as the new prior:

Two positives push belief from $9\%$ to about $91\%$. Evidence compounds.

Why does “at least one boy” give $\tfrac13$ but “the older is a boy” give $\tfrac12$?

Solution. Conditioning removes outcomes unevenly. “Older is a boy” eliminates $GG$ and $GB$, leaving two equally likely worlds $BB,BG$. “At least one boy” eliminates only $GG$, leaving three worlds $BB,BG,GB$, and $BB$ is just one of them. The information “at least one” is weaker --- it does not pin down which child --- so it cuts the space less. Intuition fails because we imagine the evidence naming a specific child when it does not.

“A family has two children; I tell you at least one is a boy born on a Tuesday. Both boys?” The Tuesday detail, seemingly irrelevant, shifts the answer to $\tfrac{13}{27}$, not $\tfrac13$.

Solution sketch. Each child is one of $14$ equally likely (gender, weekday) types. Counting the ordered pairs containing at least one “boy-Tuesday” gives $27$; those with two boys give $13$; hence $\tfrac{13}{27}\approx 0.481$. The lesson: the more specifically the evidence identifies a child, the closer the answer creeps to $\tfrac12$. Ambiguous wording has no single “right” answer until the sampling process is fixed.

A disease has prevalence $P(D)=\tfrac{1}{1000}$. A test has sensitivity $P(+\mid D)=0.99$ and false-positive rate $P(+\mid D^c)=0.01$, giving likelihood ratio $\mathrm{LR}=\tfrac{0.99}{0.01}=99$. Three independent tests all read positive. Find $P(D\mid {+}{+}{+})$.

Solution. Prior odds $=\dfrac{1/1000}{999/1000}=\dfrac{1}{999}$. Multiply by $\mathrm{LR}$ once per positive:

Convert back:

One positive left belief near $9\%$ (the base rate dominated); the odds form shows why each extra positive multiplies the odds by $99$, so three positives overwhelm even a $1$-in-$1000$ prior.

Same test. A patient tests positive, then negative on an independent retest. What is $P(D)$ now?

Solution. The negative test has its own likelihood ratio $\dfrac{P(-\mid D)}{P(-\mid D^c)}=\dfrac{0.01}{0.99}=\dfrac{1}{99}$. Chain both:

The two tests exactly cancel, returning belief to the prior $\tfrac{1}{1000}$. Contradictory evidence multiplies to a net ratio of $1$ --- a fact almost invisible in the fraction form but obvious in odds.

Weather follows a chain on states (Sunny, Rainy) with

(a) If today is sunny, probability it is rainy the day after tomorrow? (b) What fraction of days are sunny in the long run?

Solution. (a) Compute the two-step matrix:

The (Sunny $\to$ Rainy) entry is $0.28$.

(b) Solve $\boldsymbol{\pi}P=\boldsymbol{\pi}$ with $\boldsymbol{\pi}=(\pi_S,\pi_R)$, $\pi_S+\pi_R=1$. The first component gives $0.8\pi_S+0.4\pi_R=\pi_S$, i.e. $0.4\pi_R=0.2\pi_S$, so $\pi_S=2\pi_R$. With $\pi_S+\pi_R=1$ we get $\pi_S=\tfrac23,\ \pi_R=\tfrac13$. Sunny two-thirds of the time.

A token sits on states $A,B,C$ with $C$ absorbing:

Starting at $A$, what is the probability of eventual absorption at $C$? (Here it is forced, so instead find the probability that the token visits $B$ before being absorbed.)

Solution. From $A$ the first move is to $B$ or $C$ with probability $\tfrac12$ each. It visits $B$ before absorption iff the very first step lands on $B$ (from $B$ the only escape is to $C$ eventually, but the “before absorption” visit is already secured). Hence $P(\text{visit }B\mid X_0=A)=\tfrac12$. To confirm absorption is certain, note $C$ is reachable from both $A$ and $B$ and is absorbing, so with probability $1$ the token is eventually trapped at $C$: powers $P^n$ push all mass into column $C$.

You start with $\$50$ and play a fair game ($p=\tfrac12$), quitting at $\$0$ or $\$100$. Probability you reach $\$100$?

Solution. With $k=50$, $N=100$, $p=q=\tfrac12$:

In a fair game the chance of hitting the target before ruin is just your current fortune as a fraction of the target --- no arithmetic beyond a ratio.

A gambler starts at $\$1$, wants to reach $\$5$, and wins each $\$1$ bet with probability $p=0.4$ (so $q=0.6$, ratio $q/p=\tfrac32$). Probability of success?

Solution. Use the biased formula with $k=1,\ N=5,\ q/p=\tfrac32$:

Only about $7.6\%$. A small per-bet disadvantage compounds brutally over the distance to the target: fighting the house edge across many rounds is nearly hopeless. (Contrast the fair game, where the same $k/N=\tfrac15=20\%$.)

Let $e$ be the expected number of fair-coin flips to first see HH. Track states by current progress: $S_0$ (no useful streak), $S_1$ (one $H$ so far). Let $a=E[\text{flips from }S_0]$, $b=E[\text{flips from }S_1]$.

Solution. First-step analysis on each state:

From the first equation $\tfrac12 a=1+\tfrac12 b\Rightarrow a=2+b$. Substitute into the second: $b=1+\tfrac12 a=1+\tfrac12(2+b)=2+\tfrac12 b$, so $\tfrac12 b=2\Rightarrow b=4$, and $a=6$. The expected number of flips is $\boxed{6}$. (The general pattern for a streak of $n$ heads is $2^{n+1}-2$.)

In the fair gambler's-ruin game ($p=\tfrac12$), let $e_k$ be the expected number of bets until hitting $0$ or $N$ from $\$k$. Find a formula.

Solution. Condition on the first bet:

Rewrite as $e_{k+1}-2e_k+e_{k-1}=-2$: the second difference is constant, so $e_k$ is quadratic. Trying $e_k=k(N-k)$ gives $(k+1)(N-k-1)-2k(N-k)+(k-1)(N-k+1)=-2$, which checks, and $e_0=e_N=0$. Hence

Starting at the midpoint $k=N/2$ the game lasts an expected $N^2/4$ bets --- surprisingly long. First-step analysis converts the recurrence directly into the answer.