Partial Derivatives
For , the partial differentiates in while treating as a CONSTANT (and vice versa for ). All the one-variable rules apply unchanged. Example: gives and .
Why freezing works: setting turns into a function of alone, , and its graph is the curve where the vertical plane slices the surface. The partial is just , the slope of that slice, so nothing new is being differentiated. For the slice is , with slope at , and indeed . The other slice, , is , with slope at , matching .
The vector points in the direction of STEEPEST increase, and its length is that steepest slope.
The directional derivative along a unit vector is the dot product — so no direction can beat , and the level curves are always perpendicular to the gradient.
The tangent plane to at is — the two partials are exactly the two tilts of the plane. For at : .
Why the two partials are the two tilts: inside the plane, moving along changes at the rate per unit of , and moving along changes it at the rate ; a non-vertical plane is fixed by one point and those two slopes. Multiplied out, the plane is . It touches the paraboloid at and stays below it everywhere, because the gap is .
Gradient DESCENT — step opposite to reduce a loss — trains essentially every neural network. The fact that is the steepest way down, proved in this topic, is the reason a billion-parameter model can be trained at all: you never need the whole landscape, only the local gradient.
For , compute , the steepest slope at , and the slope in the direction .
Show the answerHide the answer
Hold constant for : the term differentiates to , so at . Hold constant for : . So . The steepest slope is its length, , walking in the direction . The direction is already a unit vector, so its slope is — less than , as every slope must be.
Linear Approximation and Differentials
Near ,
The differential is this change, computed on the tangent plane instead of on the surface.
Why it is accurate: the tangent plane matches the surface to first order, so the error is small compared with the step — for a smooth , halving the step roughly quarters the error. That makes the differential the working tool for two jobs: estimating a value near a point where everything is easy, and tracking how uncertainty in the inputs spreads to the output. In both, you differentiate once and plug in the small changes.
Estimate . Let and expand around , where exactly and the partials and are clean.
The steps are and , so . The true value is , so the estimate is off by about . The whole trick is the choice of base point: is close to the target and makes and both partials exact.
A pendulum gives . The length is m and the period s. The differential is , and dividing by gives .
The signs of the measurement errors are unknown, so take the worst case, where both terms push the same way: the relative error is at most . The measured value is , so m/s. The period enters squared, so its error counts twice: to improve the experiment, time more swings rather than measure the string more carefully.
Letting errors cancel. Keeping the minus sign in and plugging in for both relative errors gives , a comforting "1% error". But the errors could just as well be and , and then both terms push the same way, giving . A second slip is using the differential far from the base point: for the tangent plane at predicts at , but . The error grows with the square of the step.
Use the linear approximation of at to estimate .
Show the answerHide the answer
At the base point, , and . The steps are and , so . The true value is , so the estimate is off by about — the two first-order changes cancelled, leaving only the small second-order part.
Level Curves and the Gradient
The level curves are the contour lines of the surface . At every point, is perpendicular to the level curve through it and points toward higher values, and its length is the steepest slope — so where contours drawn at equal steps of crowd together, the gradient is large. In three variables the same holds for level surfaces .
Why perpendicular: does not change along a level curve, so the directional derivative along the curve's tangent is , and says exactly that . Why the spacing: to climb from the contour to the contour at the steepest rate takes a distance of about , so a large gradient packs the contours tightly.
Let and take the point , where , so the level curve through it is the ellipse . The gradient is .
The tangent to the ellipse at must be perpendicular to , and is, since . Implicit differentiation agrees: gives , the slope of . Walking straight toward the origin, , the rate of change is . The steepest descent rate is larger, , in the direction . On a bowl that is not round, the steepest way down does not point at the bottom.
Let at . Each partial holds the other two variables fixed: , with .
The rate of change in the direction of needs the unit vector : . And since is perpendicular to the level surface , it is the normal of the tangent plane there: , that is, .
Using a direction that is not a unit vector. With and the direction , the product is three times too big, because . The warning sign is that exceeds , the largest slope in any direction — no genuine directional derivative can do that.
For , find the rate of change at in the direction of the point , and the direction of fastest increase.
Show the answerHide the answer
The gradient is at . The direction toward is , with unit vector , so the rate is . The fastest increase is along — straight in the direction — at the rate .
Proofs & Why It Matters
For a unit vector , the slope in direction is , where is the angle between and (this dot-product form follows from the chain rule applied to ).
Since with equality only at , the slope is maximized precisely when points along , and the maximal slope is .
Walk along a level curve with constant. Differentiate both sides in (chain rule): . The tangent to the level curve is always orthogonal to the gradient — which is why contour maps and steepest-descent paths cross at right angles.
The plane through with slopes in the -direction and in the -direction is by construction.
Differentiability of at says exactly that minus this plane is — the plane matches the surface to first order, which is what makes it THE tangent plane and the basis of linear approximation.
Going Deeper: Explanations & Worked Problems
The definition mirrors one-variable calculus exactly: — wiggle only, watch the output, divide, take the limit. Geometrically, slice the surface with the vertical plane ; the slice is an ordinary curve, and is its slope.
That is why every one-variable rule (product, quotient, chain) transfers verbatim: during the computation, is a constant like any other. Two subtleties are worth knowing early. First, the mixed partials usually agree: for smooth functions (Clairaut's theorem), so the order of differentiation is a free choice. Second, having both partials is weaker than true differentiability — the honest statement is that when the partials exist and are continuous, the tangent-plane approximation has error shrinking faster than , which is what every application (linearization, error propagation, Newton's method) actually uses. For at the slice is , and the difference quotient is : slope for , for , and in the limit.
Let at the point .
Step 1 — partials: (treat constant) and (treat constant).
Step 2 — evaluate: .
Step 3 — steepest ascent: the maximum slope at this point is , achieved walking in the direction of .
Step 4 — slope in a GIVEN direction (note , required!): .
Step 5 — coherence check: ✓, as no direction can beat the gradient direction; and walking perpendicular to , e.g. along , gives slope — that is the direction the level curve runs.
Find the tangent plane to at , then use it to estimate .
Step 1 — partials: and at the point.
Step 2 — assemble the plane: .
Step 3 — estimate: .
Step 4 — compare with the truth: . The linear estimate missed by — and the miss is exactly the curvature the plane cannot see: . For this surface the tangent-plane error is ALWAYS , a perfect illustration of "error shrinks like the square of the step."