6

Let $V$ be an $n \times n$ symmetric, positive definite matrix (of rank $n$). Let $X$ be an $n \times p$ matrix of rank $p$.

Define $A^- = (A^\top A)^{-1} A^\top$ as the pseudo inverse of $A$ when $A$ is of full column rank. Note that $V^- = V^{-1}$ because $V$ is invertible.

I'd like to prove that

$$ (VX)^- = X^- V^{-1} $$

but the only theorem I know about the pseudo-inverses of products requires that both of the matrices be of the same rank AND that the second matrix has full row rank. (To wit: If $B$ is an $m \times r$ matrix of rank $r$ and $C$ is an $r \times m$  matrix of rank $r$, then $(BC)^- = C^-B^-$.)

There is likely something obvious I'm missing. Any clues?

Nathan VanHoudnos
  • 163

1 Answers1

9

I am assuming that by a "pseudoinverse" you mean Moore–Penrose pseudoinverse $A^+$ of a matrix $A$. Let us check the defining properties of the Moore-Penrose pseudoinverse against $X^+ V^{-1}$:

  1. $(VX) (X^+ V^{-1}) (VX) = VX X^+ X = VX$. Ok.
  2. $(X^+ V^{-1}) (VX) (X^+ V^{-1}) = X^+ X X^+ V^{-1} = X^+ V^{-1}$. Ok.
  3. $((VX) (X^+ V^{-1}))^* = V^{-*} (XX^+)^* V^* = V^{-2} (VX)(X^+ V^{-1}) V^2$. Hmmm...
  4. $((X^+ V^{-1}) (VX))^* = (X^+X)^* = X^+X = (X^+ V^{-1}) (VX)$. Ok.

So, the above is O.K. if and only if item 3 is O.K., i.e.,

$$((VX) (X^+ V^{-1}))^* = V^{-2} (VX)(X^+ V^{-1}) V^2.$$

However, this is not generally true. For example (by Pedro Milet in comments),

$$V = \begin{bmatrix} 2 & 1 \\ 1 & 1 \end{bmatrix}, \quad X = \begin{bmatrix} 1 \\ 0 \end{bmatrix}.$$

Then

$$(VX)^+ = \frac{1}{5} \begin{bmatrix} 2 & 1 \end{bmatrix} \ne \begin{bmatrix} 1 & -1 \end{bmatrix} = X^+ V^{-1}.$$

Notice, however, that it would work if $V$ was unitary, instead of positive definite.

Community
  • 1
Vedran Šego
  • 11,640
  • 1
    Your counterexample for $X$ is a $2 \times 2$ matrix of rank $1$ (which violates his hypotheses). – Pedro M. Aug 01 '13 at 17:32
  • 1
    But indeed, $$V =\begin{bmatrix} 2 & 1 \ 1 & 1 \end{bmatrix}, X = \begin{bmatrix} 1 \ 0 \end{bmatrix}$$ is a counter-example.

    However, I also don't see why it would work if $V$ were unitary. Isn't $$V = \frac{1}{\sqrt{2}}\begin{bmatrix} 1 & 1 \ -1 & 1 \end{bmatrix}$$ (with the same $X$ as above) a counter-example?

     – Pedro M. Aug 01 '13 at 17:50
  • Huh. I guess the reason I was having trouble was that it was false! – Nathan VanHoudnos Aug 01 '13 at 18:08
  • @PedroMilet I've missed that condition. Thank you for the correction. – Vedran Šego Aug 01 '13 at 18:35
  • @PedroMilet The unitarity... if $V$ is unitary, then $V^{-} = V$ and $V^ = V^{-1}$, so step 3 should be O.K., and I think I've used no special properties of $V$ elsewhere. – Vedran Šego Aug 01 '13 at 18:42
  • 2
    Condition 3 also works if $X^\top V = \lambda X^\top$ (every column of $X$ is an eigenvector of $V$ with the same eigenvalue). That happens to be the case in the situation I was originally studying. Thanks for your help in figuring out that the result is not general. – Nathan VanHoudnos Aug 01 '13 at 18:49
  • @VedranŠego You're right! It does follow immediately for unitary $V$, and the $V$ I mentioned is (logically) no counter-example. Thanks! – Pedro M. Aug 01 '13 at 19:28
  • 1
    Does this mean that the 3rd property in the list of Other properties of an invertible matrix on Wikipedia is incorrect? – Confounded Jan 10 '18 at 11:24
  • @Confounded Yes, Pedro M.'s example that I've used is a counterexample for that claim. The problem with this property on Wikipedia is that $V$ and/or $X$ should satisfy some conditions (see Products subsection here), which an invertible matrix and a vector don't always satisfy. – Vedran Šego Jan 11 '18 at 10:20