Differentiating Through Dependencies
When and both and depend on , the total rate of change adds one contribution per route of dependence: .
Draw the dependency tree ( on top, and below, at the bottom): multiply derivatives ALONG each path, add ACROSS paths. The rule extends to any number of variables and any depth of nesting — partials with respect to several inputs just select which bottom variable you follow. Intuition: a small change perturbs by and by ; the tangent-plane approximation converts each into a -change, and to first order the two effects simply add.
Training a neural network is nothing but this chain rule applied through thousands of nested dependencies: the loss depends on outputs, which depend on weights through layer after layer. "Multiply along paths, add across paths" is precisely the backpropagation algorithm — the multivariable chain rule is arguably the most economically important formula in modern computing.
A curve given implicitly by has slope wherever — no solving for , ever. Derivation in one line: differentiate by the chain rule: .
This retires the ad-hoc implicit differentiation of first-year calculus: compute two partials, form the ratio, attach a minus sign. In three variables, a surface has by the same argument, and is the surface normal — tangent planes to implicit surfaces come free.
The folium of Descartes is the curve . Find the slope at the point .
Show the answerHide the answer
First check that the point is on the curve: . (Errors in implicit differentiation are often a point that is not on the curve at all.) Write . Then and at the point, so . The curve is rising through , as the figure suggests.
Several Inputs, Several Paths
If and , , then
In matrix form, .
Why: hold fixed and let vary. Then and move along a curve, and the one-parameter chain rule from the first section applies with playing the role of time. The matrix form shows what the rule really says: a derivative is a linear map, and the derivative of a composition is the product of the matrices — the same statement as in one variable, one size up.
Let with and . Find and at . There and , so and ; also , and .
So : arrives through and through . Check by substituting first: , and . In the same way , and directly .
Let with , , . There are three routes from to , so three terms: .
Substituting, , which is at . Check: along the curve , whose derivative is .
For with , , the rule gives and .
Take , so and : then . Directly, , and gives . Notice that is the directional derivative straight away from the origin: the chain rule has rediscovered the gradient.
Following only one route. For , , at , writing misses the that arrives through , because depends on too. Draw the tree and make sure every route from the top to contributes a term. For the same reason, cannot be simplified by "cancelling" .
Let with and . Find at .
Show the answerHide the answer
At , and . The outer partials are and ; the inner ones are and . So . Check by substituting: , and .
Implicit Differentiation in Practice
If defines as a function of and near a point where , then
For a curve , , and is normal to the curve.
Why: hold fixed and differentiate with respect to . There are two routes from to — directly, and through — so the chain rule gives , and solving gives the formula. The condition is exactly what allows the solving: where the surface stands vertical and is not locally a function of and . The implicit function theorem makes this precise.
Find the slope of at . The point is on the curve: . With , and , so .
The gradient is normal to the curve, so the tangent line is , that is, . Solving the curve for would take the quadratic formula; the implicit route took two partial derivatives.
The surface passes through , since . With : , and , so .
So near , . At this predicts , and the true root of agrees to five decimal places. The value of was never solved for in general — a cubic — yet its rate of change came out in one line.
For one mole of an ideal gas, . Each variable is a function of the other two: , , (the subscript names the variable held fixed).
Multiply: , using . The product is , not the that "cancelling" would suggest, because each factor holds a different variable fixed. It is for every relation : each factor is , the 's cancel in pairs, and remains.
Dropping the minus sign: for the tilted ellipse, although the figure shows the curve falling through . The other classic slip is to differentiate as if were a constant: from , writing loses the terms and . The correct derivative is , which gives — the same .
The surface passes through the origin. Find and there, and the tangent plane.
Show the answerHide the answer
Check the point: . Put everything on one side, . Then , and at the origin, so and . The tangent plane is , that is, .
Proofs & Why It Matters
Differentiability means , where shrinks faster than . Divide by : .
As the difference quotients converge to , and because is sub-linear in the displacement. What survives is exactly the chain rule. SIGNIFICANCE: every related-rates problem, every implicit derivative, every Jacobian change of variables, and every gradient-descent step is this limit argument reused.
Suppose for all (homogeneity of degree ). Differentiate BOTH sides with respect to — the left by the chain rule: ; the right: . Set : .
SIGNIFICANCE: in economics this says total output equals the sum of inputs paid their marginal products (constant returns to scale); in thermodynamics it generates the Gibbs–Duhem relation — one chain-rule differentiation, two sciences.
Going Deeper: Worked Problems
The radius of a cylinder grows at cm/s while its height shrinks at cm/s. How fast is the volume changing when and ?
Step 1 — depends on two moving inputs, so .
Step 2 — partials: and at the given moment.
Step 3 — plug in the rates: cm/s.
Step 4 — interpret: the radius term dominates because volume is quadratic in but only linear in — the chain rule made that visible as the sizes of the two partials.
Find the tangent plane to the sphere at .
Step 1 — write the surface as ; the normal to a level surface is the gradient .
Step 2 — evaluate: , or just after scaling.
Step 3 — the plane through the point with that normal: , i.e. .
Step 4 — sanity: for a sphere centered at the origin the normal at a point IS the point's position vector, and on the right equals — both facts confirm the plane without redoing anything.