
Linear Algebra Linear Algebra Math Forum 
 LinkBack  Thread Tools  Display Modes 
December 8th, 2015, 10:21 PM  #1 
Newbie Joined: Oct 2014 From: China Posts: 12 Thanks: 2  Inverse problem of covariance matrix – diagonalization of Hermitian operator
(I had posted the question elsewhere but got no reply) I have understood the two things respectively: 1. Use a set of observations to construct a covariance matrix, and then compute the eigenvectors of the matrix. 2. The diagonalization the Hermitian operator $A=PGP^T$. The columns of $P$ are eigenvectors. The diagonals of $G$ are eigenvalues. However, how the two processes are related? Consider a toy problem, or an inverse problem of covariance matrix: A covariance matrix is given: $$ \left(\begin{array}{ccc} 4 & 2 & 0 & 0 \\ 2& 4& 2 & 0\\ 0 & 2 & 4 & 2\\ 0 & 0 & 2 & 4\\ \end{array}\right) $$ Can we figure out the possible observations that lead to this covariance matrix? One possible solution is: 1st observation: $(1,1,1,1)$ 2nd observation: $(\frac{1\sqrt{3}}{2},\frac{1+\sqrt{3}}{2},\frac{1+\sqrt{3}}{2},\frac{1\sqrt{3}}{2})$ 3rdobservation $(\frac{1+\sqrt{2}}{3},\frac{1+\sqrt{2}}{3},\frac{1\sqrt{2}}{3},\frac{1\sqrt{2}}{3})$ 4th observation: $(\frac{\sqrt{3}}{2\sqrt{2}}+\frac{\sqrt{10}}{4},\frac{\sqrt{3}}{2\sqrt{2}}\frac{\sqrt{10}}{4},\frac{\sqrt{3}}{2\sqrt{2}}+\fr ac{\sqrt{10}}{4},\frac{\sqrt{3}}{2\sqrt{2}}\frac{\sqrt{10}}{4})$ The pattern resembles the Fourier series. That’s why I am wondering how the construction of covariance matrix and the matrix diagonlization are somehow connected. On the other hand, one can write $A=PGP^T$ as follows: $$ A=\left(\begin{array}{cc} x_1 & x_2 & x_3 \\ y_1 & y_2 & x_3 \\ z_1 & z_2 & z_3 \\ \end{array}\right) \left(\begin{array}{cc} \lambda_1 & 0 & 0 \\ 0 & \lambda_2 & 0 \\ 0 & 0 & \lambda_3 \\ \end{array}\right) \left(\begin{array}{cc} x_1 & y_1 & z_1 \\ x_2 & y_2 & z_2 \\ x_3 & y_3 & z_3 \\ \end{array}\right)\\ =\left(\begin{array}{cc} \lambda_1 x_1^2+\lambda_2 x_2^2 +\lambda_3 x_3^2& \lambda_1 x_1y_1+\lambda_2 x_2y_2 +\lambda_3 x_3y_3 & \lambda_1 x_1z_1+\lambda_2 x_2z_2 +\lambda_3 x_3z_3 \\ \lambda_1 x_1y_1+\lambda_2 x_2y_2 +\lambda_3 x_3y_3 & \lambda_1 y_1^2+\lambda_2 y_2^2 +\lambda_3 y_3^2 & \lambda_1 y_1z_1+\lambda_2 y_2z_2 +\lambda_3 y_3z_3 \\ \lambda_1 x_1z_1+\lambda_2 x_2z_2 +\lambda_3 x_3z_3 & \lambda_1 y_1z_1+\lambda_2 y_2z_2 +\lambda_3 y_3z_3 & \lambda_1 z_1^2+\lambda_2 z_2^2 +\lambda_3 z_3^2 \\ \end{array}\right) $$ Can one interpret these values as observations in a context of covariance? Last edited by whitegreen; December 8th, 2015 at 10:25 PM. 
February 17th, 2016, 08:49 AM  #2 
Math Team Joined: Jan 2015 From: Alabama Posts: 3,261 Thanks: 894 
I am not sure what you are asking. The problem you post says "Use a set of observations to construct a covariance matrix, and then compute the eigenvectors of the matrix" but you start with the covariance matrix and try to go back to a "set of observations". I don't believe that is possible. There may be many different "sets of observations" that give the same covariance matrix.

February 19th, 2016, 05:50 AM  #3 
Senior Member Joined: Feb 2012 Posts: 144 Thanks: 16 
It is easy to generate a sample having approximately as variance matrix a given matrix (provided that this matrix is symmetric and definite positive). It A is this matrix, just write the Choleski decomposition A=HH' , then generate a standard multivariate gaussian sample of size n, so that you get an nxd matrix X (assuming A is dxd) and put Y=XH. Then VarY is approximately A (converges to A when n tends to infinity).


Tags 
–, covariance, diagonalization, hermitian, inverse, matrix, operator, problem 
Thread Tools  
Display Modes  

Similar Threads  
Thread  Thread Starter  Forum  Replies  Last Post 
Diagonalization of a big scary matrix  ulrichthegreat  Linear Algebra  1  October 6th, 2014 01:51 AM 
Orthogonal projections and matrix diagonalization  david940  Linear Algebra  0  June 29th, 2014 05:32 AM 
Closedform determinant of hermitian banded toeplitz matrix!  phd_student  Linear Algebra  0  January 1st, 2013 01:53 AM 
Cross Covariance Operator  diegomoreno  Real Analysis  0  November 28th, 2012 02:42 PM 
questions on matrix diagonalization  excellents  Linear Algebra  0  October 17th, 2009 09:12 AM 