5-1 특이값 분해

본 글은 주재걸교수님의 인공지능을 위한 선형대수 강의를 듣고 정리한 내용입니다.

 

Singular Value Decomposition

• 직사각행렬에 대한 decomposition
• 
• 𝑈 ∈ ℝ𝑚×𝑚 , 𝑉 ∈ ℝ𝑛×𝑛 : matrices with orthonormal columns, providing an orthonormal basis of Col 𝐴 and Row 𝐴, respectively
• • Σ ∈ ℝ𝑚×𝑛 : a diagonal matrix whose entries are in a decreasing order, i.e., 𝜎1 ≥ 𝜎2 ≥ ⋯ ≥ 𝜎min (𝑚,n) 

  

  특이값 분해(SVD)는 고유값 분해(eigendecomposition)처럼 행렬을 대각화하는 한 방법이다. 그런데, 특이값 분해가 유용한 이유는 행렬이 정방행렬이든 아니든 관계없이 모든 m x n 행렬에 대해 적용 가능하기 때문이다.

 

Another Perspective of SVD

• {𝐮1, … , 𝐮𝑛} for Col 𝐴 and {𝐯1, … , 𝐯𝑛} for Row 𝐴, by using, say, Gram–Schmidt orthogonalization
• 
• 𝐴𝑉 = 𝑈Σ ⟺ [𝐴𝐯1 𝐴𝐯2 ⋯ 𝐴𝐯𝑛] = [𝜎1𝐮1 𝜎2𝐮2 ⋯ 𝜎𝑛𝐮𝑛]
• 𝑉 −1 = 𝑉^𝑇 since 𝑉 ∈ ℝ𝑛×𝑛 has orthonormal columns.
• Thus 𝐴𝑉 = 𝑈Σ ⟺ 𝐴 = 𝑈Σ𝑉^T

•  Can we find the following? (Yes)

1. Orthogonal eigenvector matrices 𝑈 and 𝑉
2. Eigenvalues in Σ^2 that are all positive
3. Eigenvalues in Σ^2 that are shared by 𝐴𝐴^𝑇 and 𝐴^𝑇𝐴

Diagonalization of  Symmentric Matrices

In general, 𝐴 ∈ ℝ𝑛×𝑛 is diagonalizable if and only if 𝑛 linearly independent eigenvectors exist.

• symmetric matrix : diagonal을 기준으로 transpose 했을 때 변하지않는 행렬, always diagonalizable
  • Orthogonally diagonalizable
  • meaning that their eigenvectors are not only linearly independent, but also orthogonal to each other
  • n개의 real eigenvalues를 가지고 있음(중복 허용, 실근)
  • 각 eigenvalue에 대한 eigenspace의 dimension은 characteristic equation의 root의 multiplicity of 𝜆 와 같다
  • The eigenspaces are mutually orthogonal. That is, eigenvectors corresponding to different eigenvalues are orthogonal.

Spectral Decomposition

Positive Definite Matrices

• 𝐴 ∈ ℝ𝑛×𝑛 is positive definite if 𝐱^𝑇𝐴𝐱 > 0, ∀𝐱 ≠ 𝟎.
• 𝐴 ∈ ℝ𝑛×𝑛 is positive semi-definite if 𝐱^𝑇𝐴𝐱 ≥ 0, ∀𝐱 ≠ 𝟎.
• 𝐴 ∈ ℝ𝑛×𝑛 is positive definite if and only if the eigenvalues of 𝐴 are all positive.

 

 

출처: https://www.edwith.org/ai251 

https://darkpgmr.tistory.com/106