Pseudo inverse matrix

Tweet

Property 1

Suppose that A is mn real matrix

If m<n, then the inverse of A^TA does not exist

If mn and if the inverse of A^TA exists

A⁺=(A^TA)^-1A^T satisfies the definition of pseudoinverse

Here, A⁺A=I holds. I is identity matrix.

A: mn, A^T: nm, A⁺: nm, A^TA: nn, I: nn

The rank of A and A⁺ is n

Property 2

Suppose that A is mn real matrix

If m>n, then the inverse of AA^T does not exist

If mn and if the inverse of AA^T exists

A⁺=A^T(AA^T)^-1 satisfies the definition of pseudoinverse

Here, AA⁺=I holds. I is identity matrix.

A: mn, A^T: nm, A⁺: nm, AA^T: mm, I: mm

The rank of A and A⁺ is m

Inverse

If A is square matrix, and if the inverse of A exists, then A⁺=A^-1 holds. For the above two A⁺, AA⁺=A⁺A=AA^-1=A^-1A=I holds. I is identity matrix.

Numerical computation

The methods like Gauss-Jordan or LU decomposition can only calculate the inverse of square non-singular matrix. Even if A is not square, A^TA and AA^Tが become square; thus, it may be possible to apply Gauss-Jordan or LU decomposition. Singular Value Decomposition (SVD) may also be used for calculating the pseudoinverse.

Pseudoinverse by Singular Value Decomposition (SVD)

Suppose A is mn matrix. If m<n, attach the row of 0 and make the size m=n, a priori. Here, mn.

A=UWV^T is supposed to be the result of SVD

Assume that the left-upper part of W has larger number, and the right-lower part of W has smaller number

If the component of W is less than a threshold, set it to be 0, and define such matrix as W'

When W'=diag(w₁,w₂,...,w_k,0,0,...,0), we define

W''=diag(1/w₁,1/w₂,...,1/w_k,0,0,...,0)

Define A⁺=VW''U^T

A⁺A is nn matrix. Left-upper kk is identity matrix. Otherwise 0.

AA⁺ is mm matrix

If the rank of A is n, A⁺ satisfies the above property 1

Even if the rank of A is less than n, A⁺ satisfies the definition of pseudoinverse

Suppose that we want to solve Ax=b. Calculate x=VW''U^Tb

If the rank of A is n, then x is the value where the error is minimum

If the rank of A is less than n, then x is the solution where the norm ||x|| is minimum

Definition of Moore-Penrose generalized matrix inverse

Given mn real matrix A, nm matrix pseudoinverse A⁺ is defined as follows

AA⁺A=A

A⁺AA⁺=A⁺

(AA⁺)^T=AA⁺

(A⁺A)^T=A⁺A

Back