Symmetric Matrices & Quadratic Forms

Geometrically: every symmetric matrix is a pure stretch along some perpendicular set of axes, no shearing, no rotation mixed in. This is the spectral theorem, and it is why symmetric matrices rule applied mathematics: covariance matrices, Hessians, stiffness matrices, and graph Laplacians are all symmetric by construction. For example $S = \begin{pmatrix}3&1\\1&3\end{pmatrix}$ has eigenvalues $4$ and $2$ with perpendicular eigenvectors $(1, 1)$ and $(1, -1)$. It sends the unit circle to an ellipse whose axes point along those two directions, with half-lengths $4$ and $2$.

Why eigenvalue signs decide: rotate coordinates to the orthonormal eigenbasis, $\mathbf x = u\mathbf q_1 + v\mathbf q_2$. Because the $\mathbf q_i$ are perpendicular unit vectors, the cross term disappears and $q = \lambda_1u^2 + \lambda_2v^2$. Two positive coefficients make a bowl, two negative a dome, and opposite signs a saddle whose level curves are hyperbolas. For $q = x^2 + 4xy + y^2$, matrix $\begin{pmatrix}1&2\\2&1\end{pmatrix}$, the eigenvalues are $3$ (along $(1, 1)$) and $-1$ (along $(1, -1)$), so $q = 3u^2 - v^2$: indefinite.

The eigenvalue test is the principal-axes formula $q = \sum\lambda_iu_i^2$. The $2\times2$ determinant test follows because $\det S = \lambda_1\lambda_2$: a negative product means opposite signs, a positive one means equal signs, and then $S_{11} = q(\mathbf e_1)$ reveals which. The leading-minor test is the same fact seen through elimination: the pivots of $S$ are ratios of consecutive leading minors, and $q$ is a weighted sum of squares with the pivots as weights.

The eigenvalues are $5$ (along $(1, 1)$) and $-1$ (along $(1, -1)$): trace $4$ ✓, product $-5$ ✓. Concrete witnesses: $q(1, 1) = 2 + 6 + 2 = 10 > 0$ and $q(1, -1) = 2 - 6 + 2 = -2 < 0$. As a surface $z = q(x, y)$ is a saddle, rising along $y = x$ and falling along $y = -x$; the critical point at the origin is neither a maximum nor a minimum.

The eigenvalues are $2$ (along $(1, 1)$) and $0$ (along $(1, -1)$), both $\ge 0$: positive semidefinite, not definite. Indeed $q = (x + y)^2$ is never negative but vanishes on the whole line $y = -x$, not just at the origin. Its level curves $q = c$ are pairs of parallel lines $x + y = \pm\sqrt c$ — a trough, not a bowl.

The third minor is $\det S = 2(4 - 1) - 1(2 - 0) + 0 = 4$. All three leading minors are positive, so $q$ is positive definite — no cubic equation for eigenvalues needed. (The eigenvalues are $2$ and $2 \pm \sqrt2$, all positive, with product $2\cdot(4 - 2) = 4$ ✓.)

In principal coordinates the curve is $\lambda_1u^2 + \lambda_2v^2 = c$; setting $v = 0$ gives $u = \pm\sqrt{c/\lambda_1}$. A large eigenvalue means $q$ climbs fast in that direction, so the level $c$ is reached sooner — the ellipse is narrow there. Scaling $\mathbf x$ by $r$ scales $q$ by $r^2$, which moves the unit-circle extremes to any circle.

Half-axes: $\sqrt{8/4} = \sqrt2$ along $(1, 1)$ and $\sqrt{8/2} = 2$ along $(1, -1)$. Check an endpoint: $\sqrt2\cdot\tfrac{1}{\sqrt2}(1, 1) = (1, 1)$, and $q(1, 1) = 3 + 2 + 3 = 8$ ✓; the other is $\sqrt2(1, -1)$, where $q = 6 - 4 + 6 = 8$ ✓. So the points of the ellipse closest to the origin are $\pm(1, 1)$, at distance $\sqrt2$, and the farthest are $\pm\sqrt2(1, -1)$, at distance $2$ — no calculus needed.

So the maximum is $4\cdot9 = 36$, at $\pm\tfrac{3}{\sqrt2}(1, 1)$, and the minimum is $2\cdot9 = 18$, at $\pm\tfrac{3}{\sqrt2}(1, -1)$. Check the maximum: at $x = y = \tfrac{3}{\sqrt2}$, $x^2 = y^2 = xy = \tfrac92$, so $q = \tfrac{27}{2} + 9 + \tfrac{27}{2} = 36$ ✓. Lagrange multipliers would produce the same two directions after a page of algebra; the eigenvectors give them at once.

So $(\lambda - \mu)(\mathbf u\cdot\mathbf v) = 0$, and since $\lambda \ne \mu$: $\mathbf u\cdot\mathbf v = 0$. $\blacksquare$ Significance: orthogonal eigenvectors mean the world decomposes into independent, non-interacting directions — normal modes of vibration, principal components of data, energy levels in quantum mechanics. The proof is three lines; the consequences fill physics departments.

An average lies between the extremes, hitting $\lambda_{\min}$ and $\lambda_{\max}$ when all weight sits on one eigenvector. $\blacksquare$ This one-line argument (the Rayleigh quotient) is how Google-scale eigenvalue problems are actually solved: maximize the quotient numerically.

Step 1 — the symmetric matrix is $\begin{pmatrix}5&-2\\-2&8\end{pmatrix}$ (half the cross term).

Step 2 — eigenvalues: trace $13$, det $36$: $\lambda^2 - 13\lambda + 36 = (\lambda-4)(\lambda-9) = 0$.

Step 3 — in the eigenbasis the equation becomes $4u^2 + 9v^2 = 36$, i.e. $\dfrac{u^2}{9} + \dfrac{v^2}{4} = 1$: an ellipse with semi-axes $3$ and $2$.

Step 4 — the axes' directions are the eigenvectors: for $\lambda = 4$, $(A - 4I)\mathbf v = 0$ gives $(2, 1)$ — the long axis points along $(2,1)$. Both eigenvalues positive confirmed the ellipse before any drawing; a negative one would have meant a hyperbola.

Step 1 — confirm it is critical: $f_x = 2x + y - 3 = 0$ and $f_y = x + 2y = 0$ at $(2,-1)$ ✓.

Step 2 — the Hessian is the constant symmetric matrix $\begin{pmatrix}2&1\\1&2\end{pmatrix}$.

Step 3 — leading minors: $2 > 0$ and $\det = 3 > 0$: positive definite, so the quadratic form near the point is a bowl and $(2,-1)$ is a local (in fact global) minimum, with $f = 4 - 2 + 1 - 6 = -3$.

Step 4 — connect: the calculus "$D = f_{xx}f_{yy} - f_{xy}^2 = 3 > 0$, $f_{xx} > 0$" test is literally the leading-minor test for positive definiteness. Two courses, one criterion.