Question

Consider a set of \(m\) linear equations in \(n\) unknowns:

We can let:

And re-write the system:

\[Ax = b\]

This reduces the problem to a matrix equation, and now solving the system amounts to finding \(A^{-1}\) (or sort of). Certain properies of the matrix \(A\) yield important information about the linear system.

Underdetermined System (\(m<n\))

When \(m<n\), the linear system is said to be underdetermined. I.e. there are fewer equations than unknowns. In this case, there are either no solutions (if the system is inconsistent) or infinite solutions. A unique solution is not possible.

Overdetermined System

When \(m>n\), the system may be overdetermined. In other words, there are more equations than unknowns. They system could be inconsistent, or some of the equations could be redundant. Statistically, you can translate these situations to ‘least squares solution’ or ‘overparametrized’.

There are many techniques to solve and analyze linear systems. Our goal is to understand the theory behind many of the built-in functions, and how they efficiently solve systems of equations.

First, let’s review some linear algebra topics: