Partial Derivatives & the Gradient

Study Sheet

Partial Derivatives & the Gradient

Slopes in every direction at once

Partial Derivatives

Tip
Freeze everything but one variable
xz1123z = x³slope 3
The slice y=1y = 1 of z=x3y2z = x^3y^2 is z=x3z = x^3; its tangent at x=1x = 1 has slope fx(1,1)=3f_x(1, 1) = 3.

For f(x,y)f(x, y), the partial fxf_x differentiates in xx while treating yy as a CONSTANT (and vice versa for fyf_y). All the one-variable rules apply unchanged. Example: f=x3y2f = x^3y^2 gives fx=3x2y2f_x = 3x^2y^2 and fy=2x3yf_y = 2x^3y.

Why freezing works: setting y=by = b turns ff into a function of xx alone, g(x)=f(x,b)g(x) = f(x, b), and its graph is the curve where the vertical plane y=by = b slices the surface. The partial fx(a,b)f_x(a, b) is just g′(a)g'(a), the slope of that slice, so nothing new is being differentiated. For f=x3y2f = x^3y^2 the slice y=1y = 1 is z=x3z = x^3, with slope 33 at x=1x = 1, and indeed fx(1,1)=3⋅12⋅12=3f_x(1, 1) = 3\cdot1^2\cdot1^2 = 3. The other slice, x=1x = 1, is z=y2z = y^2, with slope 22 at y=1y = 1, matching fy(1,1)=2⋅13⋅1=2f_y(1, 1) = 2\cdot1^3\cdot1 = 2.

Concept
The gradient points uphill

The vector ∇f=(fx,fy)\nabla f = (f_x, f_y) points in the direction of STEEPEST increase, and its length is that steepest slope.

The directional derivative along a unit vector u\mathbf{u} is the dot product Duf=∇f⋅uD_{\mathbf u}f = \nabla f \cdot \mathbf u — so no direction can beat ∣∇f∣|\nabla f|, and the level curves are always perpendicular to the gradient.

Example
Tangent planes
(1, 2, 5)
Grid lines of z=x2+y2z = x^2 + y^2 and the tangent plane z=2x+4y−5z = 2x + 4y - 5 touching it at (1,2,5)(1, 2, 5) (heights drawn at 14\tfrac14 scale).

The tangent plane to z=f(x,y)z = f(x,y) at (a,b)(a, b) is z=f(a,b)+fx(a,b)(x−a)+fy(a,b)(y−b)z = f(a,b) + f_x(a,b)(x-a) + f_y(a,b)(y-b) — the two partials are exactly the two tilts of the plane. For z=x2+y2z = x^2 + y^2 at (1,2,5)(1,2,5): z=5+2(x−1)+4(y−2)z = 5 + 2(x-1) + 4(y-2).

Why the two partials are the two tilts: inside the plane, moving along y=2y = 2 changes zz at the rate fx=2f_x = 2 per unit of xx, and moving along x=1x = 1 changes it at the rate fy=4f_y = 4; a non-vertical plane is fixed by one point and those two slopes. Multiplied out, the plane is z=2x+4y−5z = 2x + 4y - 5. It touches the paraboloid at (1,2,5)(1, 2, 5) and stays below it everywhere, because the gap is x2+y2−(2x+4y−5)=(x−1)2+(y−2)2≥0x^2 + y^2 - (2x + 4y - 5) = (x - 1)^2 + (y - 2)^2 \ge 0.

Side note
Significance: the gradient is how machines learn

Gradient DESCENT — step opposite ∇f\nabla f to reduce a loss — trains essentially every neural network. The fact that −∇f-\nabla f is the steepest way down, proved in this topic, is the reason a billion-parameter model can be trained at all: you never need the whole landscape, only the local gradient.

Try it
Try it: a two-line gradient

For f(x,y)=x2+xyf(x, y) = x^2 + xy, compute ∇f(2,1)\nabla f(2, 1), the steepest slope at (2,1)(2, 1), and the slope in the direction (0,1)(0, 1).

Show the answer

Hold yy constant for fxf_x: the term xyxy differentiates to yy, so fx=2x+y=5f_x = 2x + y = 5 at (2,1)(2, 1). Hold xx constant for fyf_y: fy=x=2f_y = x = 2. So ∇f(2,1)=(5,2)\nabla f(2, 1) = (5, 2). The steepest slope is its length, ∣∇f∣=25+4=29≈5.39|\nabla f| = \sqrt{25 + 4} = \sqrt{29} \approx 5.39, walking in the direction (5,2)(5, 2). The direction (0,1)(0, 1) is already a unit vector, so its slope is ∇f⋅(0,1)=2\nabla f\cdot(0, 1) = 2 — less than 29\sqrt{29}, as every slope must be.

Linear Approximation and Differentials

Tip
The tangent plane as a calculator

Near (a,b)(a, b),

f(a+Δx, b+Δy)≈f(a,b)+fx(a,b) Δx+fy(a,b) Δy.f(a + \Delta x,\ b + \Delta y) \approx f(a, b) + f_x(a, b)\,\Delta x + f_y(a, b)\,\Delta y.

The differential df=fx dx+fy dydf = f_x\,dx + f_y\,dy is this change, computed on the tangent plane instead of on the surface.

Why it is accurate: the tangent plane matches the surface to first order, so the error is small compared with the step — for a smooth ff, halving the step roughly quarters the error. That makes the differential the working tool for two jobs: estimating a value near a point where everything is easy, and tracking how uncertainty in the inputs spreads to the output. In both, you differentiate once and plug in the small changes.

Example
Example: a square root by hand

Estimate 3.022+3.972\sqrt{3.02^2 + 3.97^2}. Let f(x,y)=x2+y2f(x, y) = \sqrt{x^2 + y^2} and expand around (3,4)(3, 4), where f=5f = 5 exactly and the partials fx=xf=35f_x = \tfrac{x}{f} = \tfrac35 and fy=yf=45f_y = \tfrac{y}{f} = \tfrac45 are clean.

The steps are Δx=0.02\Delta x = 0.02 and Δy=−0.03\Delta y = -0.03, so f≈5+35(0.02)+45(−0.03)=5+0.012−0.024=4.988f \approx 5 + \tfrac35(0.02) + \tfrac45(-0.03) = 5 + 0.012 - 0.024 = 4.988. The true value is 24.8813≈4.98812\sqrt{24.8813} \approx 4.98812, so the estimate is off by about 0.00010.0001. The whole trick is the choice of base point: (3,4)(3, 4) is close to the target and makes ff and both partials exact.

Example
Example: how measurement errors spread

A pendulum gives g=4π2LT2g = \dfrac{4\pi^2L}{T^2}. The length is L=1.00±0.01L = 1.00 \pm 0.01 m and the period T=2.00±0.02T = 2.00 \pm 0.02 s. The differential is dg=4π2T2 dL−8π2LT3 dTdg = \dfrac{4\pi^2}{T^2}\,dL - \dfrac{8\pi^2L}{T^3}\,dT, and dividing by gg gives dgg=dLL−2 dTT\dfrac{dg}{g} = \dfrac{dL}{L} - 2\,\dfrac{dT}{T}.

The signs of the measurement errors are unknown, so take the worst case, where both terms push the same way: the relative error is at most 0.011+2⋅0.022=1%+2%=3%\tfrac{0.01}{1} + 2\cdot\tfrac{0.02}{2} = 1\% + 2\% = 3\%. The measured value is g=4π2⋅14=π2≈9.87g = 4\pi^2\cdot\tfrac14 = \pi^2 \approx 9.87, so g=9.87±0.30g = 9.87 \pm 0.30 m/s2^2. The period enters squared, so its error counts twice: to improve the experiment, time more swings rather than measure the string more carefully.

Side note
Where it goes wrong

Letting errors cancel. Keeping the minus sign in dgg=dLL−2dTT\tfrac{dg}{g} = \tfrac{dL}{L} - 2\tfrac{dT}{T} and plugging in +1%+1\% for both relative errors gives 1%−2%=−1%1\% - 2\% = -1\%, a comforting "1% error". But the errors could just as well be +1%+1\% and −1%-1\%, and then both terms push gg the same way, giving 3%3\%. A second slip is using the differential far from the base point: for f=x2+y2f = x^2 + y^2 the tangent plane at (1,2)(1, 2) predicts 5+2+4=115 + 2 + 4 = 11 at (2,3)(2, 3), but f(2,3)=13f(2, 3) = 13. The error 2=12+122 = 1^2 + 1^2 grows with the square of the step.

Try it
Try it: a quick estimate

Use the linear approximation of f(x,y)=xeyf(x, y) = xe^y at (2,0)(2, 0) to estimate f(2.1,−0.05)f(2.1, -0.05).

Show the answer

At the base point, f(2,0)=2f(2, 0) = 2, fx=ey=1f_x = e^y = 1 and fy=xey=2f_y = xe^y = 2. The steps are Δx=0.1\Delta x = 0.1 and Δy=−0.05\Delta y = -0.05, so f≈2+1(0.1)+2(−0.05)=2+0.1−0.1=2f \approx 2 + 1(0.1) + 2(-0.05) = 2 + 0.1 - 0.1 = 2. The true value is 2.1 e−0.05=1.99758…2.1\,e^{-0.05} = 1.99758\ldots, so the estimate is off by about 0.00240.0024 — the two first-order changes cancelled, leaving only the small second-order part.

Level Curves and the Gradient

Tip
Reading a contour map

The level curves f(x,y)=cf(x, y) = c are the contour lines of the surface z=f(x,y)z = f(x, y). At every point, ∇f\nabla f is perpendicular to the level curve through it and points toward higher values, and its length is the steepest slope — so where contours drawn at equal steps of cc crowd together, the gradient is large. In three variables the same holds for level surfaces f(x,y,z)=cf(x, y, z) = c.

Why perpendicular: ff does not change along a level curve, so the directional derivative along the curve's tangent u\mathbf u is 00, and Duf=∇f⋅u=0D_{\mathbf u}f = \nabla f\cdot\mathbf u = 0 says exactly that ∇f⊥u\nabla f\perp\mathbf u. Why the spacing: to climb from the contour cc to the contour c+Δcc + \Delta c at the steepest rate ∣∇f∣|\nabla f| takes a distance of about Δc∣∇f∣\tfrac{\Delta c}{|\nabla f|}, so a large gradient packs the contours tightly.

Example
Example: an elliptical bowl
xy∇f8
Level curves x2+4y2=2,4,8,12x^2 + 4y^2 = 2, 4, 8, 12. At (2,1)(2, 1) the gradient is perpendicular to the ellipse; the dashed arrow points at the origin.

Let f(x,y)=x2+4y2f(x, y) = x^2 + 4y^2 and take the point (2,1)(2, 1), where f=8f = 8, so the level curve through it is the ellipse x2+4y2=8x^2 + 4y^2 = 8. The gradient is ∇f=(2x,8y)=(4,8)\nabla f = (2x, 8y) = (4, 8).

The tangent to the ellipse at (2,1)(2, 1) must be perpendicular to (4,8)(4, 8), and (2,−1)(2, -1) is, since 8−8=08 - 8 = 0. Implicit differentiation agrees: 2x+8yy′=02x + 8yy' = 0 gives y′=−x4y=−12y' = -\tfrac{x}{4y} = -\tfrac12, the slope of (2,−1)(2, -1). Walking straight toward the origin, u=−15(2,1)\mathbf u = -\tfrac{1}{\sqrt5}(2, 1), the rate of change is ∇f⋅u=−8+85=−165≈−7.16\nabla f\cdot\mathbf u = -\tfrac{8 + 8}{\sqrt5} = -\tfrac{16}{\sqrt5} \approx -7.16. The steepest descent rate is larger, ∣∇f∣=80=45≈8.94|\nabla f| = \sqrt{80} = 4\sqrt5 \approx 8.94, in the direction −(1,2)-(1, 2). On a bowl that is not round, the steepest way down does not point at the bottom.

Example
Example: a gradient in three variables

Let f(x,y,z)=xyzf(x, y, z) = xyz at (1,2,3)(1, 2, 3). Each partial holds the other two variables fixed: ∇f=(yz,xz,xy)=(6,3,2)\nabla f = (yz, xz, xy) = (6, 3, 2), with ∣∇f∣=36+9+4=7|\nabla f| = \sqrt{36 + 9 + 4} = 7.

The rate of change in the direction of (2,−1,2)(2, -1, 2) needs the unit vector 13(2,−1,2)\tfrac13(2, -1, 2): Duf=12−3+43=133D_{\mathbf u}f = \tfrac{12 - 3 + 4}{3} = \tfrac{13}{3}. And since ∇f\nabla f is perpendicular to the level surface xyz=6xyz = 6, it is the normal of the tangent plane there: 6(x−1)+3(y−2)+2(z−3)=06(x - 1) + 3(y - 2) + 2(z - 3) = 0, that is, 6x+3y+2z=186x + 3y + 2z = 18.

Side note
Where it goes wrong

Using a direction that is not a unit vector. With ∇f=(6,3,2)\nabla f = (6, 3, 2) and the direction (2,−1,2)(2, -1, 2), the product ∇f⋅(2,−1,2)=13\nabla f\cdot(2, -1, 2) = 13 is three times too big, because ∣(2,−1,2)∣=3|(2, -1, 2)| = 3. The warning sign is that 1313 exceeds ∣∇f∣=7|\nabla f| = 7, the largest slope in any direction — no genuine directional derivative can do that.

Try it
Try it: which way, and how fast?

For f(x,y)=x2−xy+3yf(x, y) = x^2 - xy + 3y, find the rate of change at (1,2)(1, 2) in the direction of the point (4,6)(4, 6), and the direction of fastest increase.

Show the answer

The gradient is ∇f=(2x−y, −x+3)=(0,2)\nabla f = (2x - y,\ -x + 3) = (0, 2) at (1,2)(1, 2). The direction toward (4,6)(4, 6) is (4,6)−(1,2)=(3,4)(4, 6) - (1, 2) = (3, 4), with unit vector 15(3,4)\tfrac15(3, 4), so the rate is 0⋅3+2⋅45=85\tfrac{0\cdot3 + 2\cdot4}{5} = \tfrac85. The fastest increase is along ∇f=(0,2)\nabla f = (0, 2) — straight in the +y+y direction — at the rate ∣∇f∣=2|\nabla f| = 2.

Proofs & Why It Matters

Tip
Proof: the gradient is the steepest direction

For a unit vector u\mathbf u, the slope in direction u\mathbf u is Duf=∇f⋅u=∣∇f∣cos⁡θD_{\mathbf u}f = \nabla f\cdot\mathbf u = |\nabla f|\cos\theta, where θ\theta is the angle between u\mathbf u and ∇f\nabla f (this dot-product form follows from the chain rule applied to t↦f(x+tu)t \mapsto f(\mathbf x + t\mathbf u)).

Reminder — The differentiation rules:  (xn)′=nxn−1,(fg)′=f′g+fg′,(fg)′=f′g−fg′g2,(f(g(x)))′=f′(g(x))g′(x)(x^{n})'=nx^{n-1},\quad (fg)'=f'g+fg',\quad \left(\tfrac{f}{g}\right)'=\frac{f'g-fg'}{g^{2}},\quad \big(f(g(x))\big)'=f'(g(x))g'(x)

Since cos⁡θ≤1\cos\theta \le 1 with equality only at θ=0\theta = 0, the slope is maximized precisely when u\mathbf u points along ∇f\nabla f, and the maximal slope is ∣∇f∣|\nabla f|. ■\blacksquare

Tip
Proof: level curves ⊥ gradient

Walk along a level curve r(t)\mathbf r(t) with f(r(t))=cf(\mathbf r(t)) = c constant. Differentiate both sides in tt (chain rule): ∇f⋅r′(t)=0\nabla f\cdot\mathbf r'(t) = 0. The tangent to the level curve is always orthogonal to the gradient — which is why contour maps and steepest-descent paths cross at right angles. ■\blacksquare

Tip
Proof: the tangent plane formula

The plane through (a,b,f(a,b))(a, b, f(a,b)) with slopes fxf_x in the xx-direction and fyf_y in the yy-direction is z=f(a,b)+fx(a,b)(x−a)+fy(a,b)(y−b)z = f(a,b) + f_x(a,b)(x-a) + f_y(a,b)(y-b) by construction.

Differentiability of ff at (a,b)(a,b) says exactly that f(x,y)f(x,y) minus this plane is o ⁣((x−a)2+(y−b)2)o\!\left(\sqrt{(x-a)^2 + (y-b)^2}\right) — the plane matches the surface to first order, which is what makes it THE tangent plane and the basis of linear approximation. ■\blacksquare

Going Deeper: Explanations & Worked Problems

Concept
Partial derivatives from the limit up
xz112345678654z = 2x²
Secants of the slice z=2x2z = 2x^2 from x=1x = 1 with h=1h = 1 and h=12h = \tfrac12 (slopes 66, 55), closing in on the tangent of slope 44.

The definition mirrors one-variable calculus exactly: fx(a,b)=lim⁡h→0f(a+h,b)−f(a,b)hf_x(a,b) = \lim_{h\to0}\dfrac{f(a+h, b) - f(a, b)}{h} — wiggle xx only, watch the output, divide, take the limit. Geometrically, slice the surface z=f(x,y)z = f(x,y) with the vertical plane y=by = b; the slice is an ordinary curve, and fxf_x is its slope.

That is why every one-variable rule (product, quotient, chain) transfers verbatim: during the computation, yy is a constant like any other. Two subtleties are worth knowing early. First, the mixed partials usually agree: fxy=fyxf_{xy} = f_{yx} for smooth functions (Clairaut's theorem), so the order of differentiation is a free choice. Second, having both partials is weaker than true differentiability — the honest statement is that when the partials exist and are continuous, the tangent-plane approximation f(a+h,b+k)≈f(a,b)+fxh+fykf(a+h, b+k) \approx f(a,b) + f_x h + f_y k has error shrinking faster than h2+k2\sqrt{h^2 + k^2}, which is what every application (linearization, error propagation, Newton's method) actually uses. For f=x2yf = x^2y at (1,2)(1, 2) the slice is g(x)=2x2g(x) = 2x^2, and the difference quotient is 2(1+h)2−2h=4+2h\dfrac{2(1 + h)^2 - 2}{h} = 4 + 2h: slope 66 for h=1h = 1, 55 for h=12h = \tfrac12, and 4=fx(1,2)4 = f_x(1, 2) in the limit.

Example
Worked: gradient, steepest slope, and a directional derivative together
xy12123∇f124
Level curves x2y=1,2,4x^2y = 1, 2, 4; at (1,2)(1, 2) the gradient (4,1)(4, 1) is perpendicular to the curve x2y=2x^2y = 2 (dashed tangent).

Let f(x,y)=x2yf(x, y) = x^2y at the point (1,2)(1, 2).

Step 1 — partials: fx=2xyf_x = 2xy (treat yy constant) and fy=x2f_y = x^2 (treat xx constant).

Step 2 — evaluate: ∇f(1,2)=(2⋅1⋅2,  12)=(4,1)\nabla f(1,2) = (2\cdot1\cdot2,\; 1^2) = (4, 1).

Step 3 — steepest ascent: the maximum slope at this point is ∣∇f∣=16+1=17≈4.12|\nabla f| = \sqrt{16 + 1} = \sqrt{17} \approx 4.12, achieved walking in the direction of (4,1)(4,1).

Step 4 — slope in a GIVEN direction u=(35,45)\mathbf u = \left(\tfrac35, \tfrac45\right) (note ∣u∣=1|\mathbf u| = 1, required!): Duf=∇f⋅u=4⋅35+1⋅45=12+45=165=3.2D_{\mathbf u}f = \nabla f\cdot\mathbf u = 4\cdot\tfrac35 + 1\cdot\tfrac45 = \tfrac{12 + 4}{5} = \tfrac{16}{5} = 3.2.

Step 5 — coherence check: 3.2<4.123.2 < 4.12 ✓, as no direction can beat the gradient direction; and walking perpendicular to (4,1)(4,1), e.g. along (1,−4)/17(1,-4)/\sqrt{17}, gives slope 4−417=0\tfrac{4 - 4}{\sqrt{17}} = 0 — that is the direction the level curve runs.

Example
Worked: a tangent plane and what it predicts

Find the tangent plane to z=x2+y2z = x^2 + y^2 at (1,2,5)(1, 2, 5), then use it to estimate f(1.1,1.9)f(1.1, 1.9).

Step 1 — partials: fx=2x=2f_x = 2x = 2 and fy=2y=4f_y = 2y = 4 at the point.

Step 2 — assemble the plane: z=5+2(x−1)+4(y−2)z = 5 + 2(x - 1) + 4(y - 2).

Step 3 — estimate: z≈5+2(0.1)+4(−0.1)=5+0.2−0.4=4.8z \approx 5 + 2(0.1) + 4(-0.1) = 5 + 0.2 - 0.4 = 4.8.

Step 4 — compare with the truth: f(1.1,1.9)=1.21+3.61=4.82f(1.1, 1.9) = 1.21 + 3.61 = 4.82. The linear estimate missed by 0.020.02 — and the miss is exactly the curvature the plane cannot see: (0.1)2+(−0.1)2=0.02(0.1)^2 + (-0.1)^2 = 0.02. For this surface the tangent-plane error is ALWAYS h2+k2h^2 + k^2, a perfect illustration of "error shrinks like the square of the step."