Module

for

Cholesky, Doolittle and Crout Factorization

Background

Definition (LU-Factorization).  The nonsingular matrix A has an LU-factorization if it can be expressed as the product of a lower-triangular matrix L and an upper triangular matrix U:

.

When this is possible we say that A has an LU-decomposition.  It turns out that this factorization (when it exists) is not unique.  If L has 1's on it's diagonal, then it is called a Doolittle factorization.  If U has 1's on its diagonal, then it is called a Crout factorization.  When    (or ),  it is called a Cholesky decomposition.

Theorem (A = LU;  Factorization with NO Pivoting).  Assume that A has a Doolittle, Crout or Cholesky factorization.  The solution X to the linear system  , is found in three steps:

1.  Construct the matrices  , if possible.

2.  Solve    for    using forward substitution.

3.  Solve    for    using back substitution.

Theorem (A = LU;  Doolittle Factorization).  Assume that A has a Doolittle factorization A = LU.
=

The solution X to the linear system
, is found in three steps:

1.  Construct the matrices  , if possible.

2.  Solve    for    using forward substitution.

3.  Solve    for    using back substitution.

For curiosity, the reader might be interested in other methods of computing  L  and  U.

Theorem (A = LU;  Crout Factorization).  Assume that A has a Crout factorization A = LU.
=

The solution X to the linear system
, is found in three steps:

1.  Construct the matrices  , if possible.

2.  Solve    for    using forward substitution.

3.  Solve    for    using back substitution.

Mathematica Subroutine (Doolittle).

Mathematica Subroutine (Crout).

Mathematica Subroutine (Forward Elimination).

Mathematica Subroutine (Back Substitution).

``````

``````

Theorem (Cholesky Factorization).  If A is real, symmetric and positive definite matrix, then it has a Cholesky factorization

where U an upper triangular matrix.

Remark.  Observe that is a lower triangular matrix, so that  A = LU.  Hence we could also write Cholesky factorization

where L a lower triangular matrix.

Theorem (A = LU;  Cholesky Factorization).  Assume that A has a Cholesky factorization  ,  where  .
=

Or if you prefer to write the Cholesky factorization as  ,  where .
=

The solution X to the linear system
, is found in three steps:

1.  Construct the matrices  , if possible.

2.  Solve    for    using forward substitution.

3.  Solve    for    using back substitution.

The following Cholesky subroutine can be used when the matrix  A  is real, symmetric and positive definite.
Observe that the loop starting with  For[j=k,j<=n,j++,  is not necessary and that  U  is computed by forming the transpose of  L.

Mathematica Subroutine (Cholesky factorization).

``````
``````

Example 1 (a).  Find the  A = LU factorization for the matrix  .  Use the Doolittle method.

Example 1 (b).  Find the A = LU  factorization for the matrix  .  Use the Crout method.

Example 1 (c).  Find the A = LU  factorization for the matrix  .  Use the Cholesky method.
Solution 1 (a).
Solution 1 (b).
Solution 1 (c).

Example 2.  Consider the linear system  .

2 (a).  Solve the linear system  AX = B  by using the Doolittle method.
2 (b).  Solve the linear system  AX = B  by using the Crout method.
2 (c).  Solve the linear system  AX = B  by using the Cholesky method.
Solution 2 (a).
Solution 2 (b).
Solution 2 (c).

Application to Polynomial Curve Fitting

Theorem (
Least-Squares Polynomial Curve Fitting). Given the    data points  ,  the least squares polynomial of degree  m  of the form

that fits the n data points is obtained by solving the following linear system

for the m+1 coefficients .  These equations are referred to as the "normal equations".

Example 3.  Find the "least squares cubic" that for the four data points.
Solution 3.

Application to Continuous Least Squares Approximation

The continuous least squares approximation to a function   on the interval [0,1] for the set of functions    can solved by using the normal equations

(1)           for   .

Where the inner product is .  Solve the linear system (1) for the coefficients and construct the approximation function

.

Definition (Gram Matrix).  The Gram matrix G is a matrix of inner products where the elements are  .

The case when the set of functions is    will produce the Hilbert matrix.  Since we require the computation to be as exact as possible and an exact formula is known for the inverse of the Hilbert matrix, this is an example where an inverse matrix comes in handy.

Example 4.  Find the continuous least squares polynomial of degree n=4 that approximates the function    over the interval  .
Solution 4.

Old Lab Project (Doolittle's Method  Doolittle's Method).  Internet hyperlinks to an old lab project.

Cholesky, Doolittle and Crout Factorizations  Cholesky, Doolittle and Crout Factorizations  Internet hyperlinks to web sites and a bibliography of articles.

(c) John H. Mathews 2004