The Chain Rule & Implicit Differentiation

Study Sheet

The Chain Rule & Implicit Differentiation

Dependency trees, implicit curves, and Euler’s theorem

Differentiating Through Dependencies

Tip
The chain rule: one term per path
∂z/∂x∂z/∂ydx/dtdy/dtzxyt
Two routes from zz down to tt: multiply along each route, then add the two products.

When z=f(x,y)z = f(x, y) and both xx and yy depend on tt, the total rate of change adds one contribution per route of dependence: dzdt=∂z∂xdxdt+∂z∂ydydt\dfrac{dz}{dt} = \dfrac{\partial z}{\partial x}\dfrac{dx}{dt} + \dfrac{\partial z}{\partial y}\dfrac{dy}{dt}.

Reminder — The differentiation rules:  (xn)′=nxn−1,(fg)′=f′g+fg′,(fg)′=f′g−fg′g2,(f(g(x)))′=f′(g(x))g′(x)(x^{n})'=nx^{n-1},\quad (fg)'=f'g+fg',\quad \left(\tfrac{f}{g}\right)'=\frac{f'g-fg'}{g^{2}},\quad \big(f(g(x))\big)'=f'(g(x))g'(x)

Draw the dependency tree (zz on top, xx and yy below, tt at the bottom): multiply derivatives ALONG each path, add ACROSS paths. The rule extends to any number of variables and any depth of nesting — partials with respect to several inputs just select which bottom variable you follow. Intuition: a small change dtdt perturbs xx by x′dtx'dt and yy by y′dty'dt; the tangent-plane approximation converts each into a zz-change, and to first order the two effects simply add.

Side note
The same rule runs backpropagation

Training a neural network is nothing but this chain rule applied through thousands of nested dependencies: the loss depends on outputs, which depend on weights through layer after layer. "Multiply along paths, add across paths" is precisely the backpropagation algorithm — the multivariable chain rule is arguably the most economically important formula in modern computing.

Tip
Implicit differentiation, industrialized

A curve given implicitly by F(x,y)=0F(x, y) = 0 has slope dydx=−FxFy\dfrac{dy}{dx} = -\dfrac{F_x}{F_y} wherever Fy≠0F_y \ne 0 — no solving for yy, ever. Derivation in one line: differentiate F(x,y(x))=0F(x, y(x)) = 0 by the chain rule: Fx+Fy y′=0F_x + F_y\,y' = 0.

This retires the ad-hoc implicit differentiation of first-year calculus: compute two partials, form the ratio, attach a minus sign. In three variables, a surface F(x,y,z)=0F(x,y,z) = 0 has ∂z∂x=−FxFz\dfrac{\partial z}{\partial x} = -\dfrac{F_x}{F_z} by the same argument, and ∇F\nabla F is the surface normal — tangent planes to implicit surfaces come free.

Try it
Try it: the folium at (2, 4)
xy(2, 4)
The folium x3+y3=9xyx^3 + y^3 = 9xy and the point (2,4)(2, 4) on its loop.

The folium of Descartes is the curve x3+y3=9xyx^3 + y^3 = 9xy. Find the slope y′y' at the point (2,4)(2, 4).

Show the answer

First check that the point is on the curve: 8+64=72=9⋅2⋅48 + 64 = 72 = 9\cdot2\cdot4. (Errors in implicit differentiation are often a point that is not on the curve at all.) Write F=x3+y3−9xyF = x^3 + y^3 - 9xy. Then Fx=3x2−9y=12−36=−24F_x = 3x^2 - 9y = 12 - 36 = -24 and Fy=3y2−9x=48−18=30F_y = 3y^2 - 9x = 48 - 18 = 30 at the point, so y′=−FxFy=−−2430=45y' = -\dfrac{F_x}{F_y} = -\dfrac{-24}{30} = \dfrac45. The curve is rising through (2,4)(2, 4), as the figure suggests.

Several Inputs, Several Paths

Tip
The chain rule through a change of variables

If z=f(x,y)z = f(x, y) and x=x(s,t)x = x(s, t), y=y(s,t)y = y(s, t), then

∂z∂s=∂z∂x∂x∂s+∂z∂y∂y∂s,∂z∂t=∂z∂x∂x∂t+∂z∂y∂y∂t.\frac{\partial z}{\partial s} = \frac{\partial z}{\partial x}\frac{\partial x}{\partial s} + \frac{\partial z}{\partial y}\frac{\partial y}{\partial s}, \qquad \frac{\partial z}{\partial t} = \frac{\partial z}{\partial x}\frac{\partial x}{\partial t} + \frac{\partial z}{\partial y}\frac{\partial y}{\partial t}.

In matrix form, (zszt)=(zxzy)(xsxtysyt)\begin{pmatrix} z_s & z_t\end{pmatrix} = \begin{pmatrix} z_x & z_y\end{pmatrix}\begin{pmatrix} x_s & x_t\\ y_s & y_t\end{pmatrix}.

Why: hold tt fixed and let ss vary. Then xx and yy move along a curve, and the one-parameter chain rule from the first section applies with ss playing the role of time. The matrix form shows what the rule really says: a derivative is a linear map, and the derivative of a composition is the product of the matrices — the same statement as (f∘g)′=f′(g) g′(f\circ g)' = f'(g)\,g' in one variable, one size up.

Example
Example: two routes to s
zxyst
zz depends on ss through xx and through yy: two routes, two terms.

Let z=x2yz = x^2y with x=s+tx = s + t and y=sty = st. Find ∂z∂s\tfrac{\partial z}{\partial s} and ∂z∂t\tfrac{\partial z}{\partial t} at (s,t)=(1,2)(s, t) = (1, 2). There x=3x = 3 and y=2y = 2, so zx=2xy=12z_x = 2xy = 12 and zy=x2=9z_y = x^2 = 9; also xs=xt=1x_s = x_t = 1, ys=t=2y_s = t = 2 and yt=s=1y_t = s = 1.

So ∂z∂s=12⋅1+9⋅2=30\tfrac{\partial z}{\partial s} = 12\cdot1 + 9\cdot2 = 30: 1212 arrives through xx and 1818 through yy. Check by substituting first: z=(s+t)2 stz = (s + t)^2\,st, and ∂z∂s=2(s+t) st+(s+t)2 t=2⋅3⋅2+9⋅2=30\tfrac{\partial z}{\partial s} = 2(s + t)\,st + (s + t)^2\,t = 2\cdot3\cdot2 + 9\cdot2 = 30. In the same way ∂z∂t=12⋅1+9⋅1=21\tfrac{\partial z}{\partial t} = 12\cdot1 + 9\cdot1 = 21, and directly 2(s+t) st+(s+t)2 s=12+9=212(s + t)\,st + (s + t)^2\,s = 12 + 9 = 21.

Example
Example: three variables along a curve

Let w=xy+z2w = xy + z^2 with x=cos⁡tx = \cos t, y=sin⁡ty = \sin t, z=tz = t. There are three routes from ww to tt, so three terms: dwdt=wxx′+wyy′+wzz′=y(−sin⁡t)+xcos⁡t+2z\dfrac{dw}{dt} = w_x x' + w_y y' + w_z z' = y(-\sin t) + x\cos t + 2z.

Substituting, dwdt=−sin⁡2t+cos⁡2t+2t=cos⁡2t+2t\dfrac{dw}{dt} = -\sin^2t + \cos^2t + 2t = \cos 2t + 2t, which is 11 at t=0t = 0. Check: along the curve w(t)=sin⁡tcos⁡t+t2=12sin⁡2t+t2w(t) = \sin t\cos t + t^2 = \tfrac12\sin 2t + t^2, whose derivative is cos⁡2t+2t\cos 2t + 2t.

Example
Example: derivatives in polar coordinates

For f(x,y)f(x, y) with x=rcos⁡θx = r\cos\theta, y=rsin⁡θy = r\sin\theta, the rule gives fr=fxcos⁡θ+fysin⁡θf_r = f_x\cos\theta + f_y\sin\theta and fθ=−fx rsin⁡θ+fy rcos⁡θf_\theta = -f_x\,r\sin\theta + f_y\,r\cos\theta.

Take f=xyf = xy, so fx=yf_x = y and fy=xf_y = x: then fr=ycos⁡θ+xsin⁡θ=2rsin⁡θcos⁡θ=rsin⁡2θf_r = y\cos\theta + x\sin\theta = 2r\sin\theta\cos\theta = r\sin 2\theta. Directly, f=r2sin⁡θcos⁡θ=12r2sin⁡2θf = r^2\sin\theta\cos\theta = \tfrac12 r^2\sin 2\theta, and ∂∂r\tfrac{\partial}{\partial r} gives rsin⁡2θr\sin 2\theta. Notice that fr=∇f⋅(cos⁡θ,sin⁡θ)f_r = \nabla f\cdot(\cos\theta, \sin\theta) is the directional derivative straight away from the origin: the chain rule has rediscovered the gradient.

Side note
Where it goes wrong

Following only one route. For z=x2yz = x^2y, x=s+tx = s + t, y=sty = st at (1,2)(1, 2), writing ∂z∂s=zx xs=12\tfrac{\partial z}{\partial s} = z_x\,x_s = 12 misses the 1818 that arrives through yy, because y=sty = st depends on ss too. Draw the tree and make sure every route from the top to ss contributes a term. For the same reason, ∂z∂x∂x∂s\tfrac{\partial z}{\partial x}\tfrac{\partial x}{\partial s} cannot be simplified by "cancelling" ∂x\partial x.

Try it
Try it: two routes to u

Let w=exyw = e^x y with x=u2x = u^2 and y=uvy = uv. Find ∂w∂u\tfrac{\partial w}{\partial u} at (u,v)=(1,3)(u, v) = (1, 3).

Show the answer

At (u,v)=(1,3)(u, v) = (1, 3), x=1x = 1 and y=3y = 3. The outer partials are wx=exy=3ew_x = e^x y = 3e and wy=ex=ew_y = e^x = e; the inner ones are xu=2u=2x_u = 2u = 2 and yu=v=3y_u = v = 3. So ∂w∂u=3e⋅2+e⋅3=9e\tfrac{\partial w}{\partial u} = 3e\cdot2 + e\cdot3 = 9e. Check by substituting: w=eu2uvw = e^{u^2}uv, and ∂w∂u=2u eu2uv+eu2v=6e+3e=9e\tfrac{\partial w}{\partial u} = 2u\,e^{u^2}uv + e^{u^2}v = 6e + 3e = 9e.

Implicit Differentiation in Practice

Tip
Implicit partial derivatives

If F(x,y,z)=0F(x, y, z) = 0 defines zz as a function of xx and yy near a point where Fz≠0F_z \ne 0, then

∂z∂x=−FxFz,∂z∂y=−FyFz.\frac{\partial z}{\partial x} = -\frac{F_x}{F_z}, \qquad \frac{\partial z}{\partial y} = -\frac{F_y}{F_z}.

For a curve F(x,y)=0F(x, y) = 0, dydx=−FxFy\dfrac{dy}{dx} = -\dfrac{F_x}{F_y}, and ∇F\nabla F is normal to the curve.

Why: hold yy fixed and differentiate F(x,y,z(x,y))=0F(x, y, z(x, y)) = 0 with respect to xx. There are two routes from FF to xx — directly, and through zz — so the chain rule gives Fx+Fz ∂z∂x=0F_x + F_z\,\tfrac{\partial z}{\partial x} = 0, and solving gives the formula. The condition Fz≠0F_z \ne 0 is exactly what allows the solving: where Fz=0F_z = 0 the surface stands vertical and zz is not locally a function of xx and yy. The implicit function theorem makes this precise.

Example
Example: a tilted ellipse
xy(4, 5)(1, 2)
The ellipse x2+xy+y2=7x^2 + xy + y^2 = 7 with its tangent line at (1,2)(1, 2) (dashed) and the normal ∇F=(4,5)\nabla F = (4, 5).

Find the slope of x2+xy+y2=7x^2 + xy + y^2 = 7 at (1,2)(1, 2). The point is on the curve: 1+2+4=71 + 2 + 4 = 7. With F=x2+xy+y2−7F = x^2 + xy + y^2 - 7, Fx=2x+y=4F_x = 2x + y = 4 and Fy=x+2y=5F_y = x + 2y = 5, so y′=−45y' = -\tfrac45.

The gradient (4,5)(4, 5) is normal to the curve, so the tangent line is 4(x−1)+5(y−2)=04(x - 1) + 5(y - 2) = 0, that is, 4x+5y=144x + 5y = 14. Solving the curve for yy would take the quadratic formula; the implicit route took two partial derivatives.

Example
Example: a surface given implicitly

The surface xyz+z3=10xyz + z^3 = 10 passes through (1,1,2)(1, 1, 2), since 2+8=102 + 8 = 10. With F=xyz+z3−10F = xyz + z^3 - 10: Fx=yz=2F_x = yz = 2, Fy=xz=2F_y = xz = 2 and Fz=xy+3z2=13F_z = xy + 3z^2 = 13, so ∂z∂x=∂z∂y=−213\tfrac{\partial z}{\partial x} = \tfrac{\partial z}{\partial y} = -\tfrac{2}{13}.

So near (1,1,2)(1, 1, 2), z≈2−213(Δx+Δy)z \approx 2 - \tfrac{2}{13}(\Delta x + \Delta y). At (1.1,1)(1.1, 1) this predicts z≈2−0.213≈1.98462z \approx 2 - \tfrac{0.2}{13} \approx 1.98462, and the true root of 1.1z+z3=101.1z + z^3 = 10 agrees to five decimal places. The value of zz was never solved for in general — a cubic — yet its rate of change came out in one line.

Example
Example: the triple product rule

For one mole of an ideal gas, PV=RTPV = RT. Each variable is a function of the other two: (∂P∂V)T=−RTV2\left(\tfrac{\partial P}{\partial V}\right)_T = -\tfrac{RT}{V^2}, (∂V∂T)P=RP\left(\tfrac{\partial V}{\partial T}\right)_P = \tfrac{R}{P}, (∂T∂P)V=VR\left(\tfrac{\partial T}{\partial P}\right)_V = \tfrac{V}{R} (the subscript names the variable held fixed).

Multiply: (−RTV2)RPVR=−RTPV=−1\left(-\tfrac{RT}{V^2}\right)\tfrac{R}{P}\tfrac{V}{R} = -\tfrac{RT}{PV} = -1, using PV=RTPV = RT. The product is −1-1, not the +1+1 that "cancelling" would suggest, because each factor holds a different variable fixed. It is −1-1 for every relation F(P,V,T)=0F(P, V, T) = 0: each factor is −FaFb-\tfrac{F_a}{F_b}, the FF's cancel in pairs, and (−1)3=−1(-1)^3 = -1 remains.

Side note
Where it goes wrong

Dropping the minus sign: y′=FxFy=+45y' = \tfrac{F_x}{F_y} = +\tfrac45 for the tilted ellipse, although the figure shows the curve falling through (1,2)(1, 2). The other classic slip is to differentiate as if yy were a constant: from x2+xy+y2=7x^2 + xy + y^2 = 7, writing 2x+y=02x + y = 0 loses the terms xy′xy' and 2yy′2yy'. The correct derivative is 2x+y+xy′+2yy′=02x + y + xy' + 2yy' = 0, which gives y′=−2x+yx+2yy' = -\tfrac{2x + y}{x + 2y} — the same −FxFy-\tfrac{F_x}{F_y}.

Try it
Try it: a surface through the origin

The surface ez+xz=y+1e^z + xz = y + 1 passes through the origin. Find ∂z∂x\tfrac{\partial z}{\partial x} and ∂z∂y\tfrac{\partial z}{\partial y} there, and the tangent plane.

Show the answer

Check the point: e0+0=0+1e^0 + 0 = 0 + 1. Put everything on one side, F=ez+xz−y−1F = e^z + xz - y - 1. Then Fx=z=0F_x = z = 0, Fy=−1F_y = -1 and Fz=ez+x=1F_z = e^z + x = 1 at the origin, so ∂z∂x=−01=0\tfrac{\partial z}{\partial x} = -\tfrac{0}{1} = 0 and ∂z∂y=−−11=1\tfrac{\partial z}{\partial y} = -\tfrac{-1}{1} = 1. The tangent plane is z=0+0⋅x+1⋅yz = 0 + 0\cdot x + 1\cdot y, that is, z=yz = y.

Proofs & Why It Matters

Tip
Proof: the chain rule from the tangent plane

Differentiability means Δz=fxΔx+fyΔy+ε\Delta z = f_x\Delta x + f_y\Delta y + \varepsilon, where ε\varepsilon shrinks faster than Δx2+Δy2\sqrt{\Delta x^2 + \Delta y^2}. Divide by Δt\Delta t: ΔzΔt=fxΔxΔt+fyΔyΔt+εΔt\dfrac{\Delta z}{\Delta t} = f_x\dfrac{\Delta x}{\Delta t} + f_y\dfrac{\Delta y}{\Delta t} + \dfrac{\varepsilon}{\Delta t}.

As Δt→0\Delta t \to 0 the difference quotients converge to x′(t),y′(t)x'(t), y'(t), and εΔt→0\tfrac{\varepsilon}{\Delta t} \to 0 because ε\varepsilon is sub-linear in the displacement. What survives is exactly the chain rule. ■\blacksquare SIGNIFICANCE: every related-rates problem, every implicit derivative, every Jacobian change of variables, and every gradient-descent step is this limit argument reused.

Tip
Proof: Euler's homogeneous-function theorem

Suppose f(tx,ty)=tnf(x,y)f(tx, ty) = t^nf(x, y) for all tt (homogeneity of degree nn). Differentiate BOTH sides with respect to tt — the left by the chain rule: x fx(tx,ty)+y fy(tx,ty)x\,f_x(tx, ty) + y\,f_y(tx, ty); the right: ntn−1f(x,y)nt^{n-1}f(x,y). Set t=1t = 1: xfx+yfy=nfxf_x + yf_y = nf.

■\blacksquare SIGNIFICANCE: in economics this says total output equals the sum of inputs paid their marginal products (constant returns to scale); in thermodynamics it generates the Gibbs–Duhem relation — one chain-rule differentiation, two sciences.

Going Deeper: Worked Problems

Example
Worked: a related-rates problem the multivariable way

The radius of a cylinder grows at 22 cm/s while its height shrinks at 11 cm/s. How fast is the volume changing when r=3r = 3 and h=5h = 5?

Step 1 — V=πr2hV = \pi r^2h depends on two moving inputs, so dVdt=Vrdrdt+Vhdhdt\dfrac{dV}{dt} = V_r\dfrac{dr}{dt} + V_h\dfrac{dh}{dt}.

Step 2 — partials: Vr=2πrh=30πV_r = 2\pi rh = 30\pi and Vh=πr2=9πV_h = \pi r^2 = 9\pi at the given moment.

Step 3 — plug in the rates: dVdt=30π(2)+9π(−1)=51π\dfrac{dV}{dt} = 30\pi(2) + 9\pi(-1) = 51\pi cm3^3/s.

Step 4 — interpret: the radius term dominates because volume is quadratic in rr but only linear in hh — the chain rule made that visible as the sizes of the two partials.

Example
Worked: an implicit surface and its tangent plane

Find the tangent plane to the sphere x2+y2+z2=14x^2 + y^2 + z^2 = 14 at (1,2,3)(1, 2, 3).

Step 1 — write the surface as F(x,y,z)=x2+y2+z2−14=0F(x,y,z) = x^2 + y^2 + z^2 - 14 = 0; the normal to a level surface is the gradient ∇F=(2x,2y,2z)\nabla F = (2x, 2y, 2z).

Step 2 — evaluate: ∇F(1,2,3)=(2,4,6)\nabla F(1,2,3) = (2, 4, 6), or just (1,2,3)(1, 2, 3) after scaling.

Step 3 — the plane through the point with that normal: 1(x−1)+2(y−2)+3(z−3)=01(x - 1) + 2(y - 2) + 3(z - 3) = 0, i.e. x+2y+3z=14x + 2y + 3z = 14.

Step 4 — sanity: for a sphere centered at the origin the normal at a point IS the point's position vector, and 1414 on the right equals ∣(1,2,3)∣2|(1,2,3)|^2 — both facts confirm the plane without redoing anything.