The Chain Rule & Implicit Differentiation

Study Sheet

The Chain Rule & Implicit Differentiation

Dependency trees, implicit curves, and Euler’s theorem

Differentiating Through Dependencies

Tip
The chain rule: one term per path

When z=f(x,y)z = f(x, y) and both xx and yy depend on tt, the total rate of change adds one contribution per route of dependence: dzdt=∂z∂xdxdt+∂z∂ydydt\dfrac{dz}{dt} = \dfrac{\partial z}{\partial x}\dfrac{dx}{dt} + \dfrac{\partial z}{\partial y}\dfrac{dy}{dt}.

Reminder — The differentiation rules:  (xn)′=nxn−1,(fg)′=f′g+fg′,(fg)′=f′g−fg′g2,(f(g(x)))′=f′(g(x))g′(x)(x^{n})'=nx^{n-1},\quad (fg)'=f'g+fg',\quad \left(\tfrac{f}{g}\right)'=\frac{f'g-fg'}{g^{2}},\quad \big(f(g(x))\big)'=f'(g(x))g'(x)

Draw the dependency tree (zz on top, xx and yy below, tt at the bottom): multiply derivatives ALONG each path, add ACROSS paths. The rule extends to any number of variables and any depth of nesting — partials with respect to several inputs just select which bottom variable you follow. Intuition: a small change dtdt perturbs xx by x′dtx'dt and yy by y′dty'dt; the tangent-plane approximation converts each into a zz-change, and to first order the two effects simply add.

Side note
The same rule runs backpropagation

Training a neural network is nothing but this chain rule applied through thousands of nested dependencies: the loss depends on outputs, which depend on weights through layer after layer. "Multiply along paths, add across paths" is precisely the backpropagation algorithm — the multivariable chain rule is arguably the most economically important formula in modern computing.

Tip
Implicit differentiation, industrialized

A curve given implicitly by F(x,y)=0F(x, y) = 0 has slope dydx=−FxFy\dfrac{dy}{dx} = -\dfrac{F_x}{F_y} wherever Fy≠0F_y \ne 0 — no solving for yy, ever. Derivation in one line: differentiate F(x,y(x))=0F(x, y(x)) = 0 by the chain rule: Fx+Fy y′=0F_x + F_y\,y' = 0.

This retires the ad-hoc implicit differentiation of first-year calculus: compute two partials, form the ratio, attach a minus sign. In three variables, a surface F(x,y,z)=0F(x,y,z) = 0 has ∂z∂x=−FxFz\dfrac{\partial z}{\partial x} = -\dfrac{F_x}{F_z} by the same argument, and ∇F\nabla F is the surface normal — tangent planes to implicit surfaces come free.

Try it
Try it: the folium at (2, 4)

For x3+y3=9xyx^3 + y^3 = 9xy find y′y' at (2,4)(2, 4). Work: F=x3+y3−9xyF = x^3 + y^3 - 9xy; Fx=3x2−9y=−24F_x = 3x^2 - 9y = -24 and Fy=3y2−9x=30F_y = 3y^2 - 9x = 30 at the point; slope =−−2430=45= -\tfrac{-24}{30} = \tfrac45. Confirm the point first (8+64=72=9⋅88 + 64 = 72 = 9\cdot8 ✓) — implicit-differentiation errors are usually really point-not-on-curve errors.

Proofs & Why It Matters

Tip
Proof: the chain rule from the tangent plane

Differentiability means Δz=fxΔx+fyΔy+ε\Delta z = f_x\Delta x + f_y\Delta y + \varepsilon, where ε\varepsilon shrinks faster than Δx2+Δy2\sqrt{\Delta x^2 + \Delta y^2}. Divide by Δt\Delta t: ΔzΔt=fxΔxΔt+fyΔyΔt+εΔt\dfrac{\Delta z}{\Delta t} = f_x\dfrac{\Delta x}{\Delta t} + f_y\dfrac{\Delta y}{\Delta t} + \dfrac{\varepsilon}{\Delta t}.

As Δt→0\Delta t \to 0 the difference quotients converge to x′(t),y′(t)x'(t), y'(t), and εΔt→0\tfrac{\varepsilon}{\Delta t} \to 0 because ε\varepsilon is sub-linear in the displacement. What survives is exactly the chain rule. ■\blacksquare SIGNIFICANCE: every related-rates problem, every implicit derivative, every Jacobian change of variables, and every gradient-descent step is this limit argument reused.

Tip
Proof: Euler's homogeneous-function theorem

Suppose f(tx,ty)=tnf(x,y)f(tx, ty) = t^nf(x, y) for all tt (homogeneity of degree nn). Differentiate BOTH sides with respect to tt — the left by the chain rule: x fx(tx,ty)+y fy(tx,ty)x\,f_x(tx, ty) + y\,f_y(tx, ty); the right: ntn−1f(x,y)nt^{n-1}f(x,y). Set t=1t = 1: xfx+yfy=nfxf_x + yf_y = nf.

■\blacksquare SIGNIFICANCE: in economics this says total output equals the sum of inputs paid their marginal products (constant returns to scale); in thermodynamics it generates the Gibbs–Duhem relation — one chain-rule differentiation, two sciences.

Going Deeper: Worked Problems

Example
Worked: a related-rates problem the multivariable way

The radius of a cylinder grows at 22 cm/s while its height shrinks at 11 cm/s. How fast is the volume changing when r=3r = 3 and h=5h = 5?

Step 1 — V=πr2hV = \pi r^2h depends on two moving inputs, so dVdt=Vrdrdt+Vhdhdt\dfrac{dV}{dt} = V_r\dfrac{dr}{dt} + V_h\dfrac{dh}{dt}.

Step 2 — partials: Vr=2πrh=30πV_r = 2\pi rh = 30\pi and Vh=πr2=9πV_h = \pi r^2 = 9\pi at the given moment.

Step 3 — plug in the rates: dVdt=30π(2)+9π(−1)=51π\dfrac{dV}{dt} = 30\pi(2) + 9\pi(-1) = 51\pi cm3^3/s.

Step 4 — interpret: the radius term dominates because volume is quadratic in rr but only linear in hh — the chain rule made that visible as the sizes of the two partials.

Example
Worked: an implicit surface and its tangent plane

Find the tangent plane to the sphere x2+y2+z2=14x^2 + y^2 + z^2 = 14 at (1,2,3)(1, 2, 3).

Step 1 — write the surface as F(x,y,z)=x2+y2+z2−14=0F(x,y,z) = x^2 + y^2 + z^2 - 14 = 0; the normal to a level surface is the gradient ∇F=(2x,2y,2z)\nabla F = (2x, 2y, 2z).

Step 2 — evaluate: ∇F(1,2,3)=(2,4,6)\nabla F(1,2,3) = (2, 4, 6), or just (1,2,3)(1, 2, 3) after scaling.

Step 3 — the plane through the point with that normal: 1(x−1)+2(y−2)+3(z−3)=01(x - 1) + 2(y - 2) + 3(z - 3) = 0, i.e. x+2y+3z=14x + 2y + 3z = 14.

Step 4 — sanity: for a sphere centered at the origin the normal at a point IS the point's position vector, and 1414 on the right equals ∣(1,2,3)∣2|(1,2,3)|^2 — both facts confirm the plane without redoing anything.