When $A$ is rectangular (more equations than unknowns) or singular, you cannot solve $Ax=b$ exactly. Instead, you find the $x$ that minimizes the error. This is the mathematical foundation of regression, curve fitting, and statistical modeling.
Such as Conjugate Gradient (CG) or GMRES . These start with a guess and refine it, making them ideal for massive systems where you only need a "good enough" solution. II. Least Squares Problems applied numerical linear algebra
Training a neural network involves massive matrix-vector multiplications. Optimizing these kernels (using libraries like BLAS or LAPACK) is what makes AI fast. When $A$ is rectangular (more equations than unknowns)
Applied Numerical Linear Algebra: The Engines of Modern Computation you cannot solve $Ax=b$ exactly. Instead