Search & glossary

Every rule and concept in Linear Algebra, with its definition and a link to the lesson that teaches it. Search by term, by idea, or by lesson title.

Glossary (71)

2×2 and 3×3 by hand: $; Determinants & Inverses › The Determinant
A = UΣVᵀ, column by column: Every $m\times n$ matrix factors as $A = U\Sigma V^{\mathsf T}$ :; Singular Values & the SVD › Computing the SVD
A gallery of maps, with their matrices derived: Derive, never memorize: a map's matrix has columns $T(\mathbf e_1), T(\mathbf e_2)$ .; Linear Transformations › Going Deeper: Explanations & Worked Problems
A matrix is a machine for systems: The system $2x + y = 7$ , $x - y = 2$ is the single equation $A\mathbf x = \mathbf b$ with $A =; Matrices & Systems › Matrix Multiplication
Absorbing chains and expected times: Some states trap you (absorbing); the natural question becomes "how long until absorption?" First-step analysis: from state $i$ , one step happens, then you are somewhere new — so $E_i = 1 + \sum_j p_{ij}E_j$ , a small linear system in the expected times.; Markov Chains & Long-Run Behavior › Probability Meets Eigenvectors
Coordinates without solving: If $\mathbf u_1, \ldots, \mathbf u_k$ are mutually orthogonal and nonzero, the projection of $\mathbf b$ onto their span is the sum of the one-dimensional projections: \[ p = $\frac{\mathbf b\cdot\mathbf u_1}{\mathbf u_1\cdot\mathbf u_1}$ u_1 + …; Orthogonality & Least Squares › Orthogonal Bases and Gram–Schmidt
Decomposing a vector: the along-and-across split: The projection formula does more than find shadows — it splits any $\mathbf u$ into two orthogonal pieces relative to a direction $\mathbf v$ : the part along, $proj_ v u = $\tfrac{\mathbf u\cdot\mathbf v}{\mathbf v\cdot\mathbf v}$ …; Orthogonality & Least Squares › Going Deeper: Explanations & Worked Problems
Diagonalization is a change of glasses: If a matrix $A$ has $n$ independent eigenvectors, pack them as the columns of $P$ and their eigenvalues into the diagonal of $D$ : then $A = PDP^{-1}$ .; Diagonalization & Matrix Powers › Changing to the Eigenbasis
Elimination as an algorithm, not an art: The method is fully mechanical.; Row Reduction & Rank › Going Deeper: Explanations & Worked Problems
Elimination records itself: When elimination needs no row swaps, it factors $A = LU$ : $U$ is the echelon form (upper triangular) and $L$ is lower triangular with $1$ s on the diagonal and, below it, the multipliers used — the entry $\ell_{ij}$ is the number $m$ in the step $R_i R…; Row Reduction & Rank › Elimination as a Factorization: A = LU
Finding eigenvalues and eigenvectors: the complete procedure: Step one: eigenvalues are the roots of $\det(A - \lambda I) = 0$ — the values of $\lambda$ making $A - \lambda I$ singular, because only a singular matrix can send a nonzero vector to zero.; Eigenvalues › Going Deeper: Explanations & Worked Problems
First-step equations for absorption probabilities: Let $h_i$ be the probability that the chain, started in state $i$ , is eventually absorbed in a target state $T$ .; Markov Chains & Long-Run Behavior › Absorbing Chains: Where You End Up
Independence testing is elimination in disguise: To decide whether vectors $\mathbf v_1, \ldots, \mathbf v_k$ are independent, ask whether $c_1\mathbf v_1 + \cdots + c_k\mathbf v_k = \mathbf 0$ forces all $c_i = 0$ — but that is a homogeneous linear system in the $c_i$ , with the vectors as columns.; Vector Spaces & Bases › Going Deeper: Explanations & Worked Problems
Invertible ⇔ nonzero determinant: $A^{-1}$ exists exactly when $\det A \ne 0$ , and for $2\times2$ : $A^-1 = $\tfrac{1}{\det A}$; Determinants & Inverses › The Determinant
Keep the big layers, drop the small ones: Write $A = \sigma_1\mathbf u_1\mathbf v_1^{\mathsf T} + \sigma_2\mathbf u_2\mathbf v_2^{\mathsf T} + \cdots$ .; Singular Values & the SVD › Low-Rank Approximation in Practice
Kernel and image: The kernel is everything sent to zero (the solution space of $A\mathbf x = \mathbf 0$ ); the image is everything hit (the span of the columns).; Linear Transformations › Maps That Respect Addition
Least squares: closest, not exact: When $A\mathbf x = \mathbf b$ has no solution, solve $A^{\mathsf T}A\,\hat{\mathbf x} = A^{\mathsf T}\mathbf b$ instead: $A\hat{\mathbf x}$ is the projection of $\mathbf b$ onto the column space — the best possible fit.; Orthogonality & Least Squares › Projection and Best Approximation
Matrices from a direction vector: Let $L$ be the line through the origin along $\mathbf v$ .; Linear Transformations › Projections and Reflections Across a Line
n steps is the nth power: Starting from the distribution $\mathbf x_0$ , after $n$ steps the distribution is $\mathbf x_n = M^n\mathbf x_0$ , and the entry $(M^n)_{ij}$ is the probability of being in state $i$ after $n$ steps, having started in state $j$ .; Markov Chains & Long-Run Behavior › Distributions Step by Step
Pivot variables in terms of free ones: Keep going past echelon form: scale each pivot to $1$ and clear the entries above it too.; Row Reduction & Rank › Reduced Echelon Form and Free Variables
Powers become easy: If $A\mathbf v = \lambda\mathbf v$ then $A^k\mathbf v = \lambda^k\mathbf v$ , and $\mathrm{tr}(A^k) = \sum\lambda_i^k$ .; Eigenvalues › Eigenvalues & Eigenvectors
Proof sketch: every basis has the same size: The exchange argument: if $\{\mathbf u_1, \ldots, \mathbf u_m\}$ is independent and $\{\mathbf b_1, \ldots, \mathbf b_n\}$ spans, insert the $\mathbf u$ s one at a time into the spanning set, each time ejecting some $\mathbf b$ (writing the new $\mathbf u$ i…; Vector Spaces & Bases › Proofs & Why It Matters
Proof: ad − bc is the area factor: The matrix sends the unit square (spanned by $\mathbf e_1, \mathbf e_2$ ) to the parallelogram spanned by the columns $(a, c)$ and $(b, d)$ .; Determinants & Inverses › Proofs & Why It Matters
Proof: Aᵏ = PDᵏP⁻¹, and why traces are power sums: $A^2 = (PDP^{-1})(PDP^{-1}) = PD(P^{-1}P)DP^{-1} = PD^2P^{-1}$ — the inner pair cancels; induction extends to all $k$ .; Diagonalization & Matrix Powers › Proofs & Why It Matters
Proof: Aᵏv = λᵏv: Induction on $k$ .; Eigenvalues › Proofs & Why It Matters
Proof: coordinates in a basis are unique: Suppose $\mathbf v = \sum a_i\mathbf b_i = \sum c_i\mathbf b_i$ in a basis $\{\mathbf b_i\}$ .; Vector Spaces & Bases › Proofs & Why It Matters
Proof: det(AB) = det A · det B (2×2, by areas): The map $\mathbf x \mapsto AB\mathbf x$ first scales areas by $\det B$ (applying $B$ ), then by $\det A$ (applying $A$ ) — so the composite scales areas by the product.; Determinants & Inverses › Proofs & Why It Matters
Proof: eigenvalues sum to the trace, multiply to the determinant (2×2): Expand the characteristic polynomial: $; Eigenvalues › Proofs & Why It Matters
Proof: eigenvectors of a symmetric matrix are orthogonal: Let $A\mathbf u = \lambda\mathbf u$ and $A\mathbf v = \mu\mathbf v$ with $\lambda \ne \mu$ .; Symmetric Matrices & Quadratic Forms › Proofs & Why It Matters
Proof: matrix multiplication is composition: Let $T_A$ and $T_B$ be the maps $\mathbf x \mapsto A\mathbf x$ and $\mathbf x \mapsto B\mathbf x$ .; Matrices & Systems › Proofs & Why It Matters
Proof: pivots decide the trichotomy: In echelon form, a row $(0;\cdots;0\,|\,b)$ with $b \ne 0$ reads $0 = b$ : no solution.; Row Reduction & Rank › Proofs & Why It Matters
Proof: rank–nullity: Row reduce $A$ ( $m \times n$ , rank $r$ ): $r$ pivot columns and $n - r$ free columns.; Row Reduction & Rank › Proofs & Why It Matters
Proof: reflections have determinant −1: A reflection fixes a line and flips the perpendicular direction.; Linear Transformations › Proofs & Why It Matters
Proof: row operations preserve solutions: Each operation is reversible: swapping rows swaps back; scaling by $c \ne 0$ undoes by $\tfrac1c$ ; adding $c$ (row $i$ ) to row $j$ undoes by subtracting.; Matrices & Systems › Proofs & Why It Matters
Proof: the 2×2 inverse formula: Multiply directly: $; Determinants & Inverses › Proofs & Why It Matters
Proof: the columns determine the map: Any $\mathbf x = x_1\mathbf e_1 + \cdots + x_n\mathbf e_n$ , so by linearity $T(\mathbf x) = x_1T(\mathbf e_1) + \cdots + x_nT(\mathbf e_n)$ — a combination of the images of the basis vectors with weights given by the coordinates.; Linear Transformations › Proofs & Why It Matters
Proof: the normal equations of least squares: The best $\hat{\mathbf x}$ minimizes $|A\mathbf x - \mathbf b|$ , i.e.; Orthogonality & Least Squares › Proofs & Why It Matters
Proof: the projection formula: Seek the multiple $c\mathbf v$ of $\mathbf v$ making the remainder $\mathbf u - c\mathbf v$ orthogonal to $\mathbf v$ : $(\mathbf u - c\mathbf v)\cdot\mathbf v = 0$ gives $c = \dfrac{\mathbf u\cdot\mathbf v}{\mathbf v\cdot\mathbf v}$ in one step.; Orthogonality & Least Squares › Proofs & Why It Matters
Proof: the range of q on the unit circle: Expand $\mathbf x$ in the orthonormal eigenbasis: $\mathbf x = \sum c_i\mathbf q_i$ with $\sum c_i^2 = 1$ on the unit circle.; Symmetric Matrices & Quadratic Forms › Proofs & Why It Matters
Proof: λ = 1 is always an eigenvalue: Let $\mathbf 1 = (1, 1, \ldots, 1)$ .; Markov Chains & Long-Run Behavior › Proofs & Why It Matters
Proof: σ₁ is the maximum stretch: Maximize $|A\mathbf x|^2 = \mathbf x^{\mathsf T}(A^{\mathsf T}A)\mathbf x$ over unit vectors $\mathbf x$ .; Singular Values & the SVD › Proofs & Why It Matters
Quadratic forms and definiteness: A quadratic form $q(\mathbf x) = \mathbf x^{\mathsf T}A\mathbf x$ (write $ax^2 + bxy + cy^2$ with the symmetric matrix $; Symmetric Matrices & Quadratic Forms › The Nicest Matrices in Mathematics
Rank and the rank–nullity theorem: The rank is the number of pivots — the true dimension of what the matrix can reach.; Row Reduction & Rank › Gaussian Elimination
Reading a matrix through its eigenvalues: Along an eigenvector, $|\lambda| > 1$ stretches, $|\lambda| < 1$ shrinks, a negative $\lambda$ flips, and $\lambda = 0$ collapses the direction to $\mathbf 0$ (it lies in the kernel).; Eigenvalues › What the Eigenvalues Tell You
Reading an ellipse off its matrix: If $S$ is positive definite with eigenvalues $\lambda_1, \lambda_2$ and unit eigenvectors $\mathbf q_1, \mathbf q_2$ , the level curve $\mathbf x^{\mathsf T}S\mathbf x = c$ is an ellipse with half-axes \[ $\sqrt{c/\lambda_1}$ along q_1, …; Symmetric Matrices & Quadratic Forms › Principal Axes and Constrained Extremes
Recurrences are matrix powers: Fibonacci's $F_{n+1} = F_n + F_{n-1}$ is the matrix iteration $F_n+1F_n =; Diagonalization & Matrix Powers › Changing to the Eigenbasis
Row times column: The $(i, j)$ entry of $AB$ is row $i$ of $A$ dotted with column $j$ of $B$ .; Matrices & Systems › Matrix Multiplication
Rows give lines, columns give combinations: For a $2\times2$ system, the row picture draws each equation as a line, and the solution is the point where the lines cross.; Matrices & Systems › The Row Picture and the Column Picture
Singular values: stretch factors for ANY matrix: Eigenvalues need square matrices and can be complex.; Singular Values & the SVD › The Master Decomposition
Solving by ratios of determinants: If $\det A \ne 0$ , the solution of $A\mathbf x = \mathbf b$ is \[x_i = $\frac{\det A_i}{\det A}$ ,\] where $A_i$ is $A$ with column $i$ replaced by $\mathbf b$ .; Determinants & Inverses › Cramer's Rule and Area Formulas
Span, independence, basis: The span of a set of vectors is everything reachable by linear combinations of them.; Vector Spaces & Bases › The Language of Vector Spaces
Stretch directions: A nonzero vector $\mathbf v$ is an eigenvector of $A$ when $A\mathbf v = \lambda\mathbf v$ : the matrix only stretches it, by the eigenvalue $\lambda$ , and never turns it off its line.; Eigenvalues › Eigenvalues & Eigenvectors
Subspaces cut dimension by conditions: Each independent linear condition on $\mathbb R^n$ removes one dimension: the hyperplane $x_1 + x_2 + x_3 + x_4 = 0$ in $\mathbb R^4$ has dimension $3$ .; Vector Spaces & Bases › The Language of Vector Spaces
The change-of-basis matrix: Put the basis vectors $\mathbf b_1, \ldots, \mathbf b_n$ as the columns of $P$ .; Vector Spaces & Bases › Change of Basis
The dominant eigenvalue wins: If $\mathbf x_0 = c_1\mathbf v_1 + \cdots + c_n\mathbf v_n$ in an eigenbasis, then \[ x_k = A^k x_0 = c_1λ_1^k v_1 + … + c_nλ_n^k v_n, u(t) = e^At u_0 = c_1e^λ_1t v_1 + …; Diagonalization & Matrix Powers › Long-Run Behavior: Powers and Differential Equations
The matrix IS the transformation: A linear map is determined by where it sends the basis vectors, and those images are the columns of its matrix.; Linear Transformations › Maps That Respect Addition
The projection formula: The projection of $\mathbf u$ onto $\mathbf v$ is \[proj_ v u = $\frac{\mathbf u\cdot\mathbf v}{\mathbf v\cdot\mathbf v}$ v,\] the shadow $\mathbf u$ casts on the line through $\mathbf v$ .; Orthogonality & Least Squares › Projection and Best Approximation
The recipe for a best-fit line: To fit $y = mx + c$ to points $(x_i, y_i)$ , $i = 1, \ldots, n$ , the normal equations $A^{\mathsf T}A\hat{\mathbf x} = A^{\mathsf T}\mathbf b$ with $A =; Orthogonality & Least Squares › Least Squares in Practice
The spectral theorem: A symmetric matrix ( $A = A^{\mathsf T}$ ) is as good as it gets: all eigenvalues real, eigenvectors of distinct eigenvalues automatically orthogonal, and a full orthonormal eigenbasis always exists — $A = Q\Lambda Q^{\mathsf T}$ with $Q$ a rotation.; Symmetric Matrices & Quadratic Forms › The Nicest Matrices in Mathematics
The subspace test: A subset $W$ of a vector space is a subspace when it contains $\mathbf 0$ and is closed under the two operations: $\mathbf u, \mathbf w \in W$ implies $\mathbf u + \mathbf w \in W$ , and $c\mathbf u \in W$ for every scalar $c$ .; Vector Spaces & Bases › Subspaces: Testing and Building
The SVD and best low-rank approximation: The singular value decomposition writes $A = \sum_i \sigma_i\,\mathbf u_i\mathbf v_i^{\mathsf T}$ : a sum of rank-one layers, ordered by importance $\sigma_1 \ge \sigma_2 \ge \cdots$ .; Singular Values & the SVD › The Master Decomposition
The two-step method, in general: Eigenvalues › Computing Eigenvectors, Beyond the Symmetric Case
Three faces of one number: The determinant answers three seemingly different questions with one number.; Determinants & Inverses › Going Deeper: Explanations & Worked Problems
Three moves solve every system: Swap two rows, scale a row by a nonzero number, add a multiple of one row to another — none of these changes the solution set.; Row Reduction & Rank › Gaussian Elimination
Three tests that agree: For a symmetric $S$ and $q(\mathbf x) = \mathbf x^{\mathsf T}S\mathbf x$ :; Symmetric Matrices & Quadratic Forms › Classifying Quadratic Forms
Transition matrices and the eigenvalue 1: A Markov chain hops among states with fixed probabilities.; Markov Chains & Long-Run Behavior › Probability Meets Eigenvectors
Two tests, and one quick disqualifier: $T$ is linear exactly when $T(\mathbf u + \mathbf v) = T\mathbf u + T\mathbf v$ and $T(c\mathbf u) = cT\mathbf u$ for all vectors and scalars.; Linear Transformations › Linear or Not?
Undo, and reverse the order: The inverse $A^{-1}$ undoes $A$ : $A^{-1}A = AA^{-1} = I$ , where $I$ has ones on the diagonal and zeros elsewhere.; Matrices & Systems › Inverses and Transposes
What a matrix does: track the basis vectors: Everything about $A\mathbf x$ is visible in two special inputs.; Matrices & Systems › Going Deeper: Explanations & Worked Problems
What each move does to det: Determinants & Inverses › Row Operations and the Determinant
When and how: $A$ ( $n\times n$ ) is diagonalizable exactly when it has $n$ independent eigenvectors.; Diagonalization & Matrix Powers › Diagonalizing Step by Step