Partial Derivatives & the Gradient

Study Sheet

Partial Derivatives & the Gradient

Slopes in every direction at once

Partial Derivatives

Tip
Freeze everything but one variable

For f(x,y)f(x, y), the partial fxf_x differentiates in xx while treating yy as a CONSTANT (and vice versa for fyf_y). All the one-variable rules apply unchanged. Example: f=x3y2f = x^3y^2 gives fx=3x2y2f_x = 3x^2y^2 and fy=2x3yf_y = 2x^3y.

Concept
The gradient points uphill

The vector ∇f=(fx,fy)\nabla f = (f_x, f_y) points in the direction of STEEPEST increase, and its length is that steepest slope.

The directional derivative along a unit vector u\mathbf{u} is the dot product Duf=∇f⋅uD_{\mathbf u}f = \nabla f \cdot \mathbf u — so no direction can beat ∣∇f∣|\nabla f|, and the level curves are always perpendicular to the gradient.

Example
Tangent planes

The tangent plane to z=f(x,y)z = f(x,y) at (a,b)(a, b) is z=f(a,b)+fx(a,b)(x−a)+fy(a,b)(y−b)z = f(a,b) + f_x(a,b)(x-a) + f_y(a,b)(y-b) — the two partials are exactly the two tilts of the plane. For z=x2+y2z = x^2 + y^2 at (1,2,5)(1,2,5): z=5+2(x−1)+4(y−2)z = 5 + 2(x-1) + 4(y-2).

Side note
Significance: the gradient is how machines learn

Gradient DESCENT — step opposite ∇f\nabla f to reduce a loss — trains essentially every neural network. The fact that −∇f-\nabla f is the steepest way down, proved in this topic, is the reason a billion-parameter model can be trained at all: you never need the whole landscape, only the local gradient.

Try it
Try it: a two-line gradient

For f(x,y)=x2+xyf(x, y) = x^2 + xy, compute ∇f(2,1)\nabla f(2, 1). Work: fx=2x+y=5f_x = 2x + y = 5, fy=x=2f_y = x = 2: ∇f=(5,2)\nabla f = (5, 2). Steepest slope: ∣∇f∣=29|\nabla f| = \sqrt{29}. And the slope heading straight along (0,1)(0,1)? ∇f⋅(0,1)=2\nabla f\cdot(0,1) = 2 — less than 29≈5.4\sqrt{29} \approx 5.4, as it must be.

Proofs & Why It Matters

Tip
Proof: the gradient is the steepest direction

For a unit vector u\mathbf u, the slope in direction u\mathbf u is Duf=∇f⋅u=∣∇f∣cos⁡θD_{\mathbf u}f = \nabla f\cdot\mathbf u = |\nabla f|\cos\theta, where θ\theta is the angle between u\mathbf u and ∇f\nabla f (this dot-product form follows from the chain rule applied to t↦f(x+tu)t \mapsto f(\mathbf x + t\mathbf u)).

Reminder — The differentiation rules:  (xn)′=nxn−1,(fg)′=f′g+fg′,(fg)′=f′g−fg′g2,(f(g(x)))′=f′(g(x))g′(x)(x^{n})'=nx^{n-1},\quad (fg)'=f'g+fg',\quad \left(\tfrac{f}{g}\right)'=\frac{f'g-fg'}{g^{2}},\quad \big(f(g(x))\big)'=f'(g(x))g'(x)

Since cos⁡θ≤1\cos\theta \le 1 with equality only at θ=0\theta = 0, the slope is maximized precisely when u\mathbf u points along ∇f\nabla f, and the maximal slope is ∣∇f∣|\nabla f|. ■\blacksquare

Tip
Proof: level curves ⊥ gradient

Walk along a level curve r(t)\mathbf r(t) with f(r(t))=cf(\mathbf r(t)) = c constant. Differentiate both sides in tt (chain rule): ∇f⋅r′(t)=0\nabla f\cdot\mathbf r'(t) = 0. The tangent to the level curve is always orthogonal to the gradient — which is why contour maps and steepest-descent paths cross at right angles. ■\blacksquare

Tip
Proof: the tangent plane formula

The plane through (a,b,f(a,b))(a, b, f(a,b)) with slopes fxf_x in the xx-direction and fyf_y in the yy-direction is z=f(a,b)+fx(a,b)(x−a)+fy(a,b)(y−b)z = f(a,b) + f_x(a,b)(x-a) + f_y(a,b)(y-b) by construction.

Differentiability of ff at (a,b)(a,b) says exactly that f(x,y)f(x,y) minus this plane is o ⁣((x−a)2+(y−b)2)o\!\left(\sqrt{(x-a)^2 + (y-b)^2}\right) — the plane matches the surface to first order, which is what makes it THE tangent plane and the basis of linear approximation. ■\blacksquare

Going Deeper: Explanations & Worked Problems

Concept
Partial derivatives from the limit up

The definition mirrors one-variable calculus exactly: fx(a,b)=lim⁡h→0f(a+h,b)−f(a,b)hf_x(a,b) = \lim_{h\to0}\dfrac{f(a+h, b) - f(a, b)}{h} — wiggle xx only, watch the output, divide, take the limit. Geometrically, slice the surface z=f(x,y)z = f(x,y) with the vertical plane y=by = b; the slice is an ordinary curve, and fxf_x is its slope.

That is why every one-variable rule (product, quotient, chain) transfers verbatim: during the computation, yy is a constant like any other. Two subtleties are worth knowing early. First, the mixed partials usually agree: fxy=fyxf_{xy} = f_{yx} for smooth functions (Clairaut's theorem), so the order of differentiation is a free choice. Second, having both partials is weaker than true differentiability — the honest statement is that when the partials exist and are continuous, the tangent-plane approximation f(a+h,b+k)≈f(a,b)+fxh+fykf(a+h, b+k) \approx f(a,b) + f_x h + f_y k has error shrinking faster than h2+k2\sqrt{h^2 + k^2}, which is what every application (linearization, error propagation, Newton's method) actually uses.

Example
Worked: gradient, steepest slope, and a directional derivative together

Let f(x,y)=x2yf(x, y) = x^2y at the point (1,2)(1, 2).

Step 1 — partials: fx=2xyf_x = 2xy (treat yy constant) and fy=x2f_y = x^2 (treat xx constant).

Step 2 — evaluate: ∇f(1,2)=(2⋅1⋅2,;12)=(4,1)\nabla f(1,2) = (2\cdot1\cdot2,; 1^2) = (4, 1).

Step 3 — steepest ascent: the maximum slope at this point is ∣∇f∣=16+1=17≈4.12|\nabla f| = \sqrt{16 + 1} = \sqrt{17} \approx 4.12, achieved walking in the direction of (4,1)(4,1).

Step 4 — slope in a GIVEN direction u=(35,45)\mathbf u = \left(\tfrac35, \tfrac45\right) (note ∣u∣=1|\mathbf u| = 1, required!): Duf=∇f⋅u=4⋅35+1⋅45=12+45=165=3.2D_{\mathbf u}f = \nabla f\cdot\mathbf u = 4\cdot\tfrac35 + 1\cdot\tfrac45 = \tfrac{12 + 4}{5} = \tfrac{16}{5} = 3.2.

Step 5 — coherence check: 3.2<4.123.2 < 4.12 ✓, as no direction can beat the gradient direction; and walking perpendicular to (4,1)(4,1), e.g. along (1,−4)/17(1,-4)/\sqrt{17}, gives slope 4−417=0\tfrac{4 - 4}{\sqrt{17}} = 0 — that is the direction the level curve runs.

Example
Worked: a tangent plane and what it predicts

Find the tangent plane to z=x2+y2z = x^2 + y^2 at (1,2,5)(1, 2, 5), then use it to estimate f(1.1,1.9)f(1.1, 1.9).

Step 1 — partials: fx=2x=2f_x = 2x = 2 and fy=2y=4f_y = 2y = 4 at the point.

Step 2 — assemble the plane: z=5+2(x−1)+4(y−2)z = 5 + 2(x - 1) + 4(y - 2).

Step 3 — estimate: z≈5+2(0.1)+4(−0.1)=5+0.2−0.4=4.8z \approx 5 + 2(0.1) + 4(-0.1) = 5 + 0.2 - 0.4 = 4.8.

Step 4 — compare with the truth: f(1.1,1.9)=1.21+3.61=4.82f(1.1, 1.9) = 1.21 + 3.61 = 4.82. The linear estimate missed by 0.020.02 — and the miss is exactly the curvature the plane cannot see: (0.1)2+(−0.1)2=0.02(0.1)^2 + (-0.1)^2 = 0.02. For this surface the tangent-plane error is ALWAYS h2+k2h^2 + k^2, a perfect illustration of "error shrinks like the square of the step."