1

I would like to cast some problems involving a symmetric hollow (diagonal entries equal to $0$) matrix in an equivalent form represented by the vectorization of the matrix. I know that if the matrix is only symmetric, then an effective way is to use the half-vectorization of the matrix and the use of duplication and elimination matrices. I would like to find something similar to the case of hollow symmetric matrices.

I have a real-valued function $f$ with argument a symmetric hollow matrix $\mathbf{A}$. Because all the information of the matrix is stored in the strictly lower (or upper) triangular part of the matrix, call it $\mathbf{A}_1$ , I guess there should be a way to rewrite the function by using the vectorization of the matrix $\mathbf{A}_1$, i.e. $\mathbf{a}=\mathbf{A}_1$.

For instance, considering a symmetric matrix $\mathbf{B}$:

$$ f(\mathbf{A})= \operatorname{trace}(\mathbf{A}\mathbf{B}) = 2 \mathbf{a}^{\top}\mathbf{b}$$

This is immediate with this particular function, but how can I generalize to other functions? For instance, what about $ f(\mathbf{A})= \operatorname{trace}(\mathbf{A}^\top\mathbf{A}\mathbf{B})$. Can in general also the product $\mathbf{A}\mathbf{A}$ be rewritten with a particular reshape of the vectors product?

Thank you

EDIT

If I have function $f(\mathbf{A})$, can I compute in this case gradient and Hessian in the hollow-half-vec space, and being able to compute back the associated matrices? As a related subquestion, is the $\operatorname{vechh(\cdot)}$ an invertible operation? Of course, once that I have $\mathbf{a}=\operatorname{vechh(\mathbf{A})}$ I can easily reshape the vector in a way to get back $\mathbf{A}$. However, how is this mathematically represented? I should be able to do something like:

$$ \mathbf{A}= \operatorname{vec}^{-1} (\mathbf{D}\mathbf{H}^{-1}\mathbf{a}) \mathbf{P}^\top= \operatorname{vec}^{-1} (\mathbf{D}\mathbf{H}^{-1}\mathbf{H}\mathbf{E}\operatorname{vec}(\mathbf{AP})) \mathbf{P}^\top $$

where $\mathbf{D}$,$ \mathbf{E}$ are the duplication and elimination matrices, respectively, and $\mathbf{H}^{-1}$ is the inverse of the "hollow" matrix $\mathbf{H}=[\mathbf{0}_{l\times n} \mathbf{I}_l]$.

As a case study, we could take $f(\mathbf{A})= \operatorname{trace}(\mathbf{A}\mathbf{B}) = 2 \mathbf{a}^{\top}\mathbf{b}$, with $\mathbf{A}$, $\mathbf{B}$ symmetric matrices, $\mathbf{A}$ hollow. It would be nice to have a methodology to express each function $f(\mathbf{A})$ in an equivalent form $f(\mathbf{a})$.

yes
  • 888
  • By "hollow", do you mean that the diagonal entries are equal to zero? – Ben Grossmann Nov 16 '20 at 17:10
  • Yes, exactly. I edited. – yes Nov 16 '20 at 18:06
  • The only way to represent the operation ${\rm vec}^{-1}(a)$ is to use third-order tensors, e.g. $$A=\mathbb M\cdot a$$ as described in this answer. – greg Nov 17 '20 at 13:03
  • Since the "hallow" matrix is rectangular it does not have an inverse. However, its pseudoinverse is $H^+=H^T,$ which works in all of your formulas. – greg Nov 17 '20 at 13:08
  • Thank you for the reference. Do you think is it "doable" to rewrite functions of the form $f(\mathbf{A})$ as $f(\mathbf{a})$? I would like to update the variable $\mathbf{a}$ in the $\operatorname{vechh}$ space by means of gradient or Newton methods. – yes Nov 17 '20 at 14:46
  • You've already answered the question, i.e. it's just function composition $f(A) = f({\rm vechh}^{-1}(a))$ – greg Nov 17 '20 at 15:38

1 Answers1

1

$\def\m#1{ \left[\begin{array}{r}#1\end{array}\right] }$ Given a hollow matrix $$\eqalign{ A &\in {\mathbb R}^{n\times n} \\ }$$ the hollow-half-vec operation is analogous to the more familiar half-vec operation. Both can be described in terms of the standard vec operator $$\eqalign{ {\rm vechh}(A) &= E_h\cdot{\rm vec}(A) \quad&\sim\quad {\rm vech}(A) &= E\cdot{\rm vec}(A) \\ {\rm vec}(A) &= D_h\cdot{\rm vechh}(A) \quad&\sim\quad \;\;{\rm vec}(A) &= D\cdot{\rm vech}(A) \\ }$$ where $E_h\in{\mathbb R}^{\ell\times n^2}$ is the hollow-elimination matrix. It is a sparse binary matrix $\,\ell = \tfrac 12(n^2-n)\,$. There is one non-zero element in each row, whose column index corresponds to the index of the conserved element in ${\rm vec}(A)$.

The hollow-duplication matrix is $D_h\in{\mathbb R}^{n^2\times\ell}$ is also a sparse binary matrix, whose elements are such that $E_hD_h=I$ and whose columns sum to two, i.e. $\;\tfrac 12D_h^T{\tt1} = {\tt1}.\;$ Again, this is analogous to the half-vec case where $ED=I\,$ (however the column sums of $D$ vary between one and two).

Interestingly, the pseudoinverse $D_h^+$ can serve as an elimination matrix, although $E_h^+$ fails as a duplication matrix, i.e. $$\eqalign{ {\rm vechh}(A) &= D_h^+\cdot{\rm vec}(A) \quad&\sim\quad {\rm vech}(A) &= D^+\cdot{\rm vec}(A) \\ {\rm vec}(A) &\ne E_h^+\cdot{\rm vechh}(A) \quad&\sim\quad \;\;{\rm vec}(A) &\ne E^+\cdot{\rm vech}(A) \\ D_h^+D_h &= I \quad&\sim\quad \quad D^+D &= I \\ }$$ $\Big[\,$In the hollow case, the calculation is particularly easy since $D_h^+ = \tfrac 12D_h^T\;\Big]$

For example, for $\;n=4,\;\ell=6$ $$\eqalign{ &E_h[1,2] &= 1,\qquad &D_h(2,1) &= 1,\quad &D_h(5,1) &= 1 \\ &E_h[2,3] &= 1,\qquad &D_h(3,2) &= 1,\quad &D_h(9,2) &= 1 \\ &E_h[3,4] &= 1,\qquad &D_h(4,3) &= 1,\quad &D_h(13,3)&= 1 \\ &E_h[4,7] &= 1,\qquad &D_h(7,4) &= 1,\quad &D_h(10,4)&= 1 \\ &E_h[5,8] &= 1,\qquad &D_h(8,5) &= 1,\quad &D_h(14,5)&= 1 \\ &E_h[6,12]&= 1,\qquad &D_h(12,6)&= 1,\quad &D_h(15,6)&= 1 \\ }$$ The indices of the vec and vechh vectors mapped onto $4\times 4$ matrices helps elucidate the components given above. $$\eqalign{ \m{1&5&9&13\\2&6&10&14\\3&7&11&15\\4&8&12&16} \qquad\qquad \m{0&1&2&3\\1&0&4&5\\2&4&0&6\\3&5&6&0} \\ }$$

greg
  • 40,553
  • Thank you greg. This relation is indeed something needed. After your comment, I edited the question, for advanced computations. – yes Nov 17 '20 at 10:56
  • My original idea with the permutation matrix and didn't work out because $(AP)$ is not a symmetric matrix, so it didn't make sense to take the half-vec of it. So I switched to the vec-operator with non-standard duplication/elimination matrices. – greg Nov 17 '20 at 21:16
  • Well, even even if it was not symmetric, the equation in my EDIT part of the question turns out to be correct (it should). That is, the multiplication for the hallow operator matrix ${\rm H}$ and the use of $\mathbf{P}^\top$ at the end of the chain achieve that. – yes Nov 17 '20 at 21:21
  • It should be $D_hE_h=I$. But still, what I say in my above comment should hold. Where do you see it wrong? – yes Nov 18 '20 at 09:24
  • I think there is a subtlety in the product $D_hE_h=I$. It turns out that indeed this matrix product acts as identity matrix when applied to the vectorization of the hollow matrix, i.e. $\operatorname{vec}(A)= D_hE_h \operatorname{vec}(A) $, but this only because it multiplies the zeros of the $\operatorname{vec}(A)$ with all-$0$ rows of $D_hE_h$, hence acting as an identity matrix for this particular case. But actually it is not. – yes Nov 18 '20 at 10:44
  • Thanks greg, I accepted this answer. Can you give me a confirm that also $D_hE_h=I$? And by the way, I did not see on the web such discussion about these computations. Are them just a consequence of matrix calculus or people already worked on it? – yes Nov 19 '20 at 17:50
  • Sorry, but $D_hE_h\ne I$ and this is also true for the standard duplication/elimination matrices, $DE\ne I$. Even if you choose $D^+$ as the elimination matrix, $DD^+\ne I$. – greg Nov 19 '20 at 19:42
  • Here's a simple way to think about it. These are rectangular matrices, so if you form the "big product" it will be rank deficient. Hence it cannot possibly equal the identity matrix. – greg Nov 19 '20 at 19:49
  • Sorry, as in my previous comment, it acts as identity matrix for $\operatorname{vec}(A)$ if $A$ is hollow, because it has zeros in the diagonal in the exact same positions of the zeros of the vectorization of $A$. This is why we can write $D_hE_h \operatorname{vec}(A) = \operatorname{vec}(A)$. The $E_h$ eliminates zeros and redundant information, while $A_h$ restores the redundant information and the zeros. Do you agree? – yes Nov 20 '20 at 09:22
  • Yes, I agree. By definition the duplication matrix must satisfy ${\rm vec}(A)=D_h\cdot{\rm vechh}(A)$ when $A$ is hollow and ${\rm vec}(A)=D\cdot{\rm vech}(A)$ when $A$ is symmetric. But to state this property as $D_hE_h=I$ is incorrect. On the other hand, since $(D_hE_h)^2=D_hE_h$, you could say that $D_hE_h$ is projector into the space of hollow-half-vec vectors. – greg Nov 20 '20 at 11:49
  • I agree. I mentioned in a previous comment: are these computations just a consequence of matrix calculus applied to these special cases or there is also a specified theory behind it? – yes Nov 20 '20 at 15:29