1. Orthogonal Projections onto Lines and Hyperplanes

The orthogonal projection of a vector v onto a 1-dimensional subspace spanned by a non-zero vector u is defined by minimizing the Euclidean distance ||v - cu|| over all real scalars c. Setting the derivative of the squared error to zero yields:

When projecting onto an arbitrary flat plane passing through the origin with normal vector n = ⟨A, B, C⟩, any vector v decomposes uniquely into a component parallel to the normal and a component parallel to the plane:

2. The Gram-Schmidt Orthonormalization Algorithm

Given a set of linearly independent vectors (v₁, v₂, ..., v_k), the Gram-Schmidt process constructs a mutually orthogonal set of vectors (u₁, u₂, ..., u_k) by iteratively subtracting the projections onto all previously constructed basis elements:

Normalizing each vector e_k = u_k / ||u_k|| yields an orthonormal basis, where e_i · e_j = 1 if i = j, and 0 otherwise. This forms the mathematical foundation of the QR matrix decomposition (A = QR), where Q is an orthogonal matrix and R is upper triangular.

3. The 4 Fundamental Subspaces of Gilbert Strang

For any real m × n matrix A with rank r, linear algebra identifies four natural, canonical subspaces:

• Row Space Row(A) in ℝⁿ: Dimension r. Spanned by the rows of A (or non-zero rows of RREF).
• Null Space Null(A) in ℝⁿ: Dimension n - r. The kernel consisting of all x such that Ax = 0.
• Column Space Col(A) in ℝᵐ: Dimension r. Spanned by the columns of A (the image of the linear map).
• Left Null Space Null(Aᵀ) in ℝᵐ: Dimension m - r. The cokernel consisting of all y such that Aᵀy = 0.

The Fundamental Theorem of Linear Algebra states that Row(A) and Null(A) are orthogonal complements in ℝⁿ, and Col(A) and Null(Aᵀ) are orthogonal complements in ℝᵐ.

1. Orthogonal Projections onto Lines and Hyperplanes

The orthogonal projection of a vector v onto a 1-dimensional subspace spanned by a non-zero vector u is defined by minimizing the Euclidean distance ||v - cu|| over all real scalars c. Setting the derivative of the squared error to zero yields:

When projecting onto an arbitrary flat plane passing through the origin with normal vector n = ⟨A, B, C⟩, any vector v decomposes uniquely into a component parallel to the normal and a component parallel to the plane:

2. The Gram-Schmidt Orthonormalization Algorithm

Given a set of linearly independent vectors (v₁, v₂, ..., v_k), the Gram-Schmidt process constructs a mutually orthogonal set of vectors (u₁, u₂, ..., u_k) by iteratively subtracting the projections onto all previously constructed basis elements:

Normalizing each vector e_k = u_k / ||u_k|| yields an orthonormal basis, where e_i · e_j = 1 if i = j, and 0 otherwise. This forms the mathematical foundation of the QR matrix decomposition (A = QR), where Q is an orthogonal matrix and R is upper triangular.

3. The 4 Fundamental Subspaces of Gilbert Strang

For any real m × n matrix A with rank r, linear algebra identifies four natural, canonical subspaces:

• Row Space Row(A) in ℝⁿ: Dimension r. Spanned by the rows of A (or non-zero rows of RREF).
• Null Space Null(A) in ℝⁿ: Dimension n - r. The kernel consisting of all x such that Ax = 0.
• Column Space Col(A) in ℝᵐ: Dimension r. Spanned by the columns of A (the image of the linear map).
• Left Null Space Null(Aᵀ) in ℝᵐ: Dimension m - r. The cokernel consisting of all y such that Aᵀy = 0.

The Fundamental Theorem of Linear Algebra states that Row(A) and Null(A) are orthogonal complements in ℝⁿ, and Col(A) and Null(Aᵀ) are orthogonal complements in ℝᵐ.