Given a non-singular matrix $A\in$ $\Bbb R^{n\times n}$ (invertible) with SVD decomposition $(U, \Sigma, V)$, how would you interpret geometrically $A$ being ill conditioned?
From what I know, $A$ is ill-conditioned when its condition number is large.
I also know that cond$(A)$ is equal to the ratio of the largest to the smallest singular value of $A$ in magnitude.
Any thoughts?
Here's one way to interpret it "geometrically":
Consider the set $S = \{x \in \Bbb R^n: \|x\|_2 = 1\}$, which is the "$n$-dimensional hypersphere" of radius $1$. The map $A$ take the sphere $S$ to some "hyper-ellipsoid" $A(S)$.
The condition number is a measurement of how "skinny" the resulting ellipsoid is (specifically, it is the ratio of the lengths of the "major axis" and "minor axis"). The worse-conditioned the matrix, the "skinnier" the ellipsoid.
If $A$ is not invertible (i.e. if its condition number is $\infty$), then the resulting ellipsoid is "flat".