Optimization & Lagrange Multipliers

Why these conditions: at an interior extremum every slice through the point has a horizontal tangent, so both partials vanish. Near such a point $f$ is governed by its quadratic part $\tfrac12(f_{xx}h^2 + 2f_{xy}hk + f_{yy}k^2)$. $D > 0$ means this quadratic keeps one sign in every direction — a bowl or a dome, and the sign of $f_{xx}$ says which — while $D < 0$ means it takes both signs. The test can overrule the slices you happen to look at: $f = x^2 + 3xy + y^2$ curves upward along both axes, yet $f_{xx} = f_{yy} = 2$, $f_{xy} = 3$ and $D = 4 - 9 = -5 < 0$. The origin is a saddle, and along the line $y = -x$, $f = x^2 - 3x^2 + x^2 = -x^2$ curves down.

Why tangency: at the best point, moving along the constraint cannot change $f$ to first order, so the constraint curve runs along a level curve of $f$ there. Two curves that touch have parallel normals, and the normals are $\nabla f$ and $\nabla g$. For $xy$ on $x + y = 10$: $\nabla(xy) = (y, x)$ and $\nabla g = (1, 1)$, so $y = \lambda = x$, and the constraint gives $x = y = 5$, $\lambda = 5$. The maximum is $25$, and the level curve $xy = 25$ touches the line at $(5, 5)$.

Why $(1, 2)$: $\nabla(x + 2y) = (1, 2)$ and $\nabla(x^2 + y^2) = (2x, 2y)$, so parallel gradients mean $(x, y)$ is a multiple of $(1, 2)$. On the circle $x^2 + y^2 = 5$ the only multiples are $\pm(1, 2)$; the plus sign gives the maximum $1 + 4 = 5$ and the minus sign the minimum $-5$. The general bound $r\sqrt{a^2 + b^2}$ is here $\sqrt5\cdot\sqrt5 = 5$.

Why the boundary needs its own search: at an edge point, $f$ may still be increasing toward the outside, where the region does not allow you to go, so the gradient does not have to vanish there. The interior test $\nabla f = \mathbf 0$ cannot see such points. Both conditions on the region matter: on the open disk $x^2 + y^2 < 1$, $f = x$ gets arbitrarily close to $1$ but never reaches a maximum.

On the edge, $4 - 2x$ is largest at $x = -2$, giving $8$ at $(-2, 0)$, and smallest at $x = 2$, giving $0$ at $(2, 0)$. Comparing the candidates $-1$, $8$, $0$: the absolute minimum is $-1$ at $(1, 0)$ and the absolute maximum is $8$ at $(-2, 0)$. A geometric check: $f = (x - 1)^2 + y^2 - 1$ is the squared distance from $(1, 0)$, minus $1$. The disk's nearest point to $(1, 0)$ is $(1, 0)$ itself, and its farthest is $(-2, 0)$, at distance $3$: $9 - 1 = 8$.

The candidate values are $-4$ inside and $0, 2, -3, -1, -2$ on the edges. The maximum, $2$, occurs at the two corners $(0, \pm1)$; the minimum, $-4$, at $(2, 0)$. Notice that the maximum is at points where no derivative of any kind vanishes: along the edge $x = 0$, $f = 2y^2$ is still increasing when it hits the corner. Without the corners, the maximum would be missed.

Why $\lambda$ is a rate: at the optimum, $\nabla f = \lambda\nabla g$. Raise the constraint level by $dc$ and the optimal point moves by a small step $d\mathbf x$ that keeps it on the new constraint, so $\nabla g\cdot d\mathbf x = dc$. The optimal value then changes by $\nabla f\cdot d\mathbf x = \lambda\,\nabla g\cdot d\mathbf x = \lambda\,dc$. Economists call $\lambda$ the shadow price of the constraint: what one more unit of budget is worth.

Put this into the constraint: $\tfrac\lambda2(1 + 4 + 4) = 9$, so $\lambda = 2$ and the point is $(1, 2, 2)$, with $f = 9$. Check with the distance formula: the plane is $\tfrac{|-9|}{3} = 3$ from the origin, and $3^2 = 9$. Check the meaning of $\lambda$: for the constraint $x + 2y + 2z = c$ the minimum is $\left(\tfrac c3\right)^2 = \tfrac{c^2}{9}$, whose derivative at $c = 9$ is $\tfrac{2c}{9} = 2 = \lambda$.

Substitute one equation into the other: $y = 2\lambda(8\lambda y) = 16\lambda^2 y$. If $y = 0$ then $x = 0$, which is not on the ellipse, so $16\lambda^2 = 1$ and $\lambda = \pm\tfrac14$. For $\lambda = \tfrac14$, $x = 2y$, and the constraint gives $4y^2 + 4y^2 = 8$, $y = \pm1$: the points $(2, 1)$ and $(-2, -1)$, where $xy = 2$. For $\lambda = -\tfrac14$, $x = -2y$: the points $(-2, 1)$ and $(2, -1)$, where $xy = -2$. The maximum is $2$ and the minimum $-2$. At $(2, 1)$, $\nabla f = (1, 2)$ and $\nabla g = (4, 8) = 4(1, 2)$.

If $D > 0$ both terms share the sign of $f_{xx}$ (a definite bowl: min or max); if $D < 0$ the two terms disagree, so $Q$ takes both signs — a saddle. $\blacksquare$

Thus $\nabla f$ is orthogonal to EVERY tangent direction of the constraint — but the vectors orthogonal to that whole tangent plane are exactly the multiples of $\nabla g$. Hence $\nabla f = \lambda\nabla g$. $\blacksquare$

But the converse fails in a way one variable never shows: at a SADDLE, the surface rises along one line and falls along another, with $\nabla f = \mathbf 0$ all the same ($f = x^2 - y^2$ at the origin is the model: uphill along the $x$-axis, downhill along the $y$-axis). That is why the $D$-test exists: near a critical point, $f$ is governed by its quadratic part $\tfrac12(f_{xx}h^2 + 2f_{xy}hk + f_{yy}k^2)$, and $D = f_{xx}f_{yy} - f_{xy}^2$ decides whether that quadratic keeps one sign (a genuine bowl or dome) or changes sign (a saddle). And remember the boundary: on a closed region, extrema can sit on the edge with $\nabla f \ne \mathbf 0$ — that is precisely the situation Lagrange multipliers handle.

Step 3 — test $(1, 0)$: $D = 12 > 0$ and $f_{xx} = 6 > 0$ — both slice curvatures upward: a LOCAL MIN, value $f(1,0) = 1 - 3 = -2$.

Step 4 — test $(-1, 0)$: $D = -12 < 0$ — a SADDLE, no extremum, even though the gradient vanishes.

Step 5 — global view: as $x \to -\infty$, $x^3$ drags $f \to -\infty$, so the local min at $(1,0)$ is NOT global — a reminder that the $D$-test speaks only about a neighborhood.

Step 3 — enforce the constraint: $x^2 + (2x)^2 = 5$ gives $5x^2 = 5$, $x = \pm1$, hence the candidates $(1, 2)$ and $(-1, -2)$.

Step 4 — evaluate $f$ at each: $f(1,2) = 1 + 4 = 5$ and $f(-1,-2) = -5$. The constraint circle is closed and bounded, so these are the global max and min: maximum $5$.

Step 5 — see it geometrically: the level lines $x + 2y = c$ are parallel lines sliding outward as $c$ grows; the largest $c$ still touching the circle touches it TANGENTLY, at the point where the radius is parallel to $(1,2)$ — which is exactly what Step 1 said.