Symmetric Matrices & Quadratic Forms

Study Sheet

Symmetric Matrices & Quadratic Forms

The spectral theorem and definiteness

The Nicest Matrices in Mathematics

Tip
The spectral theorem

A SYMMETRIC matrix (A=ATA = A^{\mathsf T}) is as good as it gets: all eigenvalues real, eigenvectors of distinct eigenvalues automatically ORTHOGONAL, and a full orthonormal eigenbasis always exists — A=QΛQTA = Q\Lambda Q^{\mathsf T} with QQ a rotation.

Geometrically: every symmetric matrix is a pure stretch along some perpendicular set of axes, no shearing, no rotation mixed in. This is the spectral theorem, and it is why symmetric matrices rule applied mathematics: covariance matrices, Hessians, stiffness matrices, and graph Laplacians are all symmetric by construction.

Tip
Quadratic forms and definiteness

A quadratic form q(x)=xTAxq(\mathbf x) = \mathbf x^{\mathsf T}A\mathbf x (write ax2+bxy+cy2ax^2 + bxy + cy^2 with the symmetric matrix (ab/2b/2c)\begin{pmatrix}a & b/2\\ b/2 & c\end{pmatrix} — HALVE the cross term) is classified by eigenvalue signs: all positive — positive definite, a bowl with a genuine minimum; all negative — a dome; mixed — a saddle. On the unit circle, qq ranges exactly over [λmin⁡,λmax⁡][\lambda_{\min}, \lambda_{\max}], attained along the eigenvectors (the PRINCIPAL AXES). The quick test without eigenvalues: positive definite iff a>0a > 0 and det⁡A>0\det A > 0. Recognize this as the honest form of the second-derivative DD-test from multivariable calculus — the Hessian is a symmetric matrix, and D>0,fxx>0D > 0, f_{xx} > 0 is exactly the 2×22\times2 definiteness test.

Side note
Ellipses are eigen-pictures

The level curve q(x)=1q(\mathbf x) = 1 of a positive-definite form is an ELLIPSE whose axes point along the eigenvectors with half-lengths 1/λi1/\sqrt{\lambda_i}. Rotate to the eigenbasis and the cross term vanishes: λ1u2+λ2v2=1\lambda_1u^2 + \lambda_2v^2 = 1. Every tilted ellipse you have ever completed the square on was a symmetric matrix asking to be diagonalized.

Try it
Try it: extremes on the unit circle

Maximize and minimize q=5x2+4xy+5y2q = 5x^2 + 4xy + 5y^2 on x2+y2=1x^2 + y^2 = 1. Work: matrix (5225)\begin{pmatrix}5&2\\2&5\end{pmatrix}; trace 1010, det 2121; eigenvalues 33 and 77. Max =7= 7 at ±12(1,1)\pm\tfrac{1}{\sqrt2}(1,1), min =3= 3 at ±12(1,−1)\pm\tfrac{1}{\sqrt2}(1,-1).

Verify: q(12,12)=52+2+52=7q\left(\tfrac1{\sqrt2},\tfrac1{\sqrt2}\right) = \tfrac52 + 2 + \tfrac52 = 7 ✓. Lagrange multipliers on a quadratic ARE the eigenvalue problem.

Proofs & Why It Matters

Tip
Proof: eigenvectors of a symmetric matrix are orthogonal

Let Au=λuA\mathbf u = \lambda\mathbf u and Av=μvA\mathbf v = \mu\mathbf v with λ≠μ\lambda \ne \mu. Compute λ(u⋅v)=(Au)⋅v=uTATv=uTAv=μ(u⋅v)\lambda(\mathbf u\cdot\mathbf v) = (A\mathbf u)\cdot\mathbf v = \mathbf u^{\mathsf T}A^{\mathsf T}\mathbf v = \mathbf u^{\mathsf T}A\mathbf v = \mu(\mathbf u\cdot\mathbf v), using symmetry in the middle.

So (λ−μ)(u⋅v)=0(\lambda - \mu)(\mathbf u\cdot\mathbf v) = 0, and since λ≠μ\lambda \ne \mu: u⋅v=0\mathbf u\cdot\mathbf v = 0. ■\blacksquare SIGNIFICANCE: orthogonal eigenvectors mean the world decomposes into independent, non-interacting directions — normal modes of vibration, principal components of data, energy levels in quantum mechanics. The proof is three lines; the consequences fill physics departments.

Tip
Proof: the range of q on the unit circle

Expand x\mathbf x in the orthonormal eigenbasis: x=∑ciqi\mathbf x = \sum c_i\mathbf q_i with ∑ci2=1\sum c_i^2 = 1 on the unit circle. Then q(x)=∑λici2q(\mathbf x) = \sum\lambda_ic_i^2 — a weighted AVERAGE of the eigenvalues with weights ci2c_i^2.

An average lies between the extremes, hitting λmin⁡\lambda_{\min} and λmax⁡\lambda_{\max} when all weight sits on one eigenvector. ■\blacksquare This one-line argument (the Rayleigh quotient) is how Google-scale eigenvalue problems are actually solved: maximize the quotient numerically.

Going Deeper: Worked Problems

Example
Worked: rotating away a cross term

Identify the conic 5x2−4xy+8y2=365x^2 - 4xy + 8y^2 = 36 by diagonalizing its quadratic form.

Step 1 — the symmetric matrix is (5−2−28)\begin{pmatrix}5&-2\\-2&8\end{pmatrix} (half the cross term).

Step 2 — eigenvalues: trace 1313, det 3636: λ2−13λ+36=(λ−4)(λ−9)=0\lambda^2 - 13\lambda + 36 = (\lambda-4)(\lambda-9) = 0.

Step 3 — in the eigenbasis the equation becomes 4u2+9v2=364u^2 + 9v^2 = 36, i.e. u29+v24=1\dfrac{u^2}{9} + \dfrac{v^2}{4} = 1: an ELLIPSE with semi-axes 33 and 22.

Step 4 — the axes' directions are the eigenvectors: for λ=4\lambda = 4, (A−4I)v=0(A - 4I)\mathbf v = 0 gives (2,1)(2, 1) — the long axis points along (2,1)(2,1). Both eigenvalues positive confirmed the ellipse before any drawing; a negative one would have meant a hyperbola.

Example
Worked: the Hessian test as a definiteness check

Classify the critical point of f(x,y)=x2+xy+y2−3xf(x,y) = x^2 + xy + y^2 - 3x at (2,−1)(2, -1).

Step 1 — confirm it is critical: fx=2x+y−3=0f_x = 2x + y - 3 = 0 and fy=x+2y=0f_y = x + 2y = 0 at (2,−1)(2,-1) ✓.

Step 2 — the Hessian is the constant symmetric matrix (2112)\begin{pmatrix}2&1\\1&2\end{pmatrix}.

Step 3 — leading minors: 2>02 > 0 and det⁡=3>0\det = 3 > 0: POSITIVE DEFINITE, so the quadratic form near the point is a bowl and (2,−1)(2,-1) is a local (in fact global) minimum, with f=4−2+1−6=−3f = 4 - 2 + 1 - 6 = -3.

Step 4 — connect: the calculus "D=fxxfyy−fxy2=3>0D = f_{xx}f_{yy} - f_{xy}^2 = 3 > 0, fxx>0f_{xx} > 0" test is literally the leading-minor test for positive definiteness. Two courses, one criterion.