29

If I have a square invertible matrix and I take its determinant, and I find that $\det(A) \approx 0$, does this imply that the matrix is poorly conditioned?

Is the converse also true? Does an ill-conditioned matrix have a nearly zero determinant?

Here is something I tried in Octave:

a = rand(4,4);
det(a) %0.008
cond(a)%125
a(:,4) = 1*a(:,1) + 2*a(:,2) = 0.000000001*ones(4,1);
det(a)%1.8E-11
cond(a)%3.46E10
Ben
  • 1,493
  • 1
  • 12
  • 29
Inquest
  • 3,394
  • 3
  • 26
  • 44

2 Answers2

38

It's the largeness of the condition number $\kappa(\mathbf A)$ that measures the nearness to singularity, not the tininess of the determinant.

For instance, the diagonal matrix $10^{-50} \mathbf I$ has tiny determinant, but is well-conditioned.

On the flip side, consider the following family of square upper triangular matrices, due to Alexander Ostrowski (and also studied by Jim Wilkinson):

$$\mathbf U=\begin{pmatrix}1&2&\cdots&2\\&1&\ddots&\vdots\\&&\ddots&2\\&&&1\end{pmatrix}$$

The determinant of the $n\times n$ matrix $\mathbf U$ is always $1$, but the ratio of the largest to the smallest singular value (i.e. the 2-norm condition number $\kappa_2(\mathbf U)=\dfrac{\sigma_1}{\sigma_n}$) was shown by Ostrowski to be equal to $\cot^2\dfrac{\pi}{4n}$, which can be seen to increase for increasing $n$.

J. M.
  • 3,155
  • 28
  • 37
  • Isn't the tininess of the determinant same as the nearness to singularity? Isn't $10^{-15}I$ nearly singular as well? – Inquest Feb 16 '12 at 13:02
  • 1
    @Nunoxic: most certainly not; before I launch into details, are you already familiar with the singular value decomposition? – J. M. Feb 16 '12 at 13:40
  • Very little. I just know that is decomposes the matrix in $U\Sigma V^*$ with U and V being orthogonal and $\Sigma$ being diagonal. – Inquest Feb 16 '12 at 13:49
  • 2
    Very good. That's all you need to know. The idea is that very important information on the conditioning is concentrated in $\mathbf \Sigma$. In particular, you will want to look for the largest and smallest values (remember that the decomposition is defined such that the diagonal entries of $\mathbf \Sigma$ are nonnegative) in that matrix's diagonal. The ratio of the largest to the smallest diagonal entry is the condition number $\kappa$. What size of condition number you should be concerned with depends on the machine your working on... – J. M. Feb 16 '12 at 13:59
  • 2
    ...but in general, when solving linear equations with that matrix, you stand to lose $\approx \log_b \kappa$ base-$b$ digits in your solution. That's a rough rule of thumb for the condition number; so if you're working with only 16 digits, a $\kappa$ of $10^13$ should be cause for concern. – J. M. Feb 16 '12 at 14:02
  • If I take the Eigenvalues of $A^TA$ and take the square root of the ratio of the max to min, will that give me the condition number as well? I mean, that is how I remember it. Also, will a zero value along the diagonal imply a singular matrix? – Inquest Feb 16 '12 at 14:04
  • Regd. the loss of digits, could you give me a reference for this? I really wanted to learn about the repercussions of high $\kappa(A)$ – Inquest Feb 16 '12 at 14:07
  • 1
    Yes, but that is not the recommended method for determining the condition number (an explanation of which is for another question). I presume you know how to invert a diagonal matrix, no? – J. M. Feb 16 '12 at 14:08
  • 2
    "Regd. the loss of digits, could you give me a reference for this?" - I could, but this really is one of those things that you should be experimenting on your own in a computing environment for reinforcement. – J. M. Feb 16 '12 at 14:10
  • 1
    +1 for the impressive upper triangular matrix example. Also, regarding the loss of digits, in addition to experimenting on one's own, one should definitely look at numerical analysis textbooks specifically devoted to the issue of lost precision due to ill-conditioning and work out problems as exercises. Actual practical experience cannot be emphasized enough here. – Geoff Oxberry Feb 17 '12 at 07:41
19

As $\det(kA)=k^n\det A$, the determinant can be made arbitrarily large or small by simple rescaling (which doesn't change the condition number). Especially in high dimensions, even scaling by an innocent factor of 2 changes the determinant by a huge amount.

Thus never use the determinant to assess condition or closeness to singularity.

On the other hand, for almost all well-posed numerical problems, the condition is closely related to the distance to singularity, in the sense of the smallest relative perturbation needed to make the problem ill-posed. In particular, this holds for linear systems.

Arnold Neumaier
  • 11,318
  • 20
  • 47