Module

for

Least Squares Polynomials

   

Theorem (Least-Squares Polynomial Curve Fitting). Given the  [Graphics:Images/LeastSqPolyMod_gr_1.gif]  data points  [Graphics:Images/LeastSqPolyMod_gr_2.gif],  the least squares polynomial of degree  m  of the form  

        [Graphics:Images/LeastSqPolyMod_gr_3.gif]

that fits the n data points is obtained by solving the following linear system


        [Graphics:Images/LeastSqPolyMod_gr_4.gif][Graphics:Images/LeastSqPolyMod_gr_5.gif]  

for the m+1 coefficients [Graphics:Images/LeastSqPolyMod_gr_6.gif].  These equations are referred to as the "normal equations".

Proof  Least Squares Polynomials  Least Squares Polynomials  

 

    One thing is certain, to find the least squares polynomial the above linear system must be solved. There are various linear system solvers that could be used for this task.  However, since this is such an important computation, most mathematical software programs have a built-in subroutine for this purpose.  In Mathematica it is called the " Fit" procedure.  Fit[data, funs, vars] finds a least­squares fit to a list of data as a linear combination of the functions funs of variables vars.

We will check the "closeness of fit" with the Root Mean Square or RMS measure for the "error in the fit."

[Graphics:Images/LeastSqPolyMod_gr_7.gif]

Computer Programs  Least Squares Polynomials  Least Squares Polynomials  

 

Mathematica Subroutine (Least Squares Parabola).

[Graphics:Images/LeastSqPolyMod_gr_8.gif]

Example 1.  Find the standard "least squares parabola"  [Graphics:Images/LeastSqPolyMod_gr_9.gif]  for the data points [Graphics:Images/LeastSqPolyMod_gr_10.gif].  
Use the subroutine LSParabola to find the line.  Compare with the line obtained with Mathematica's Fit procedure.
Solution 1.

 

Example 2.  Find the polynomial curve fit of degree = 2  for the points [Graphics:Images/LeastSqPolyMod_gr_38.gif].
Use Mathematica to find the "Least Square Quadratic", and find the RMS error.
Solution 2.

 

Example 3.  Find the polynomial curve fit of degree = 3  for the points [Graphics:Images/LeastSqPolyMod_gr_52.gif].
Use Mathematica to find the "Least Square Cubic", and find the RMS error.
Solution 3.

 

Example 4.  Find the polynomial curve fit of degree = 4  for the points [Graphics:Images/LeastSqPolyMod_gr_60.gif].
Use Mathematica to find the "Least Square Quartic", and find the RMS error.
Solution 4.

 

Example 5.  Find the polynomial curve fit of degree = 5  for the points [Graphics:Images/LeastSqPolyMod_gr_68.gif].
Use Mathematica to find the "Least Square Quintic", and find the RMS error.
Solution 5.

 

Example 6.  Why is the RMS error for  [Graphics:Images/LeastSqPolyMod_gr_76.gif]  essentially zero ?
Solution 6.

 

 

Caution for polynomial curve fitting.   

    Something goes radically wrong if the data is radically "NOT polynomial."  This phenomenon is called "polynomial wiggle."  The next example illustrates this concept.  

 

Example 7.  Find the least squares polynomial fits of degree n = 2, 3, 4, 5 for the points [Graphics:Images/LeastSqPolyMod_gr_86.gif].
Solution 7.

 

 

Linear Least Squares

    The linear least-squares problem is stated as follows.  Suppose that [Graphics:Images/LeastSqPolyMod_gr_104.gif] data points  [Graphics:Images/LeastSqPolyMod_gr_105.gif]  and a set of  [Graphics:Images/LeastSqPolyMod_gr_106.gif] linearly independent functions  [Graphics:Images/LeastSqPolyMod_gr_107.gif] are given.  We want to fine  [Graphics:Images/LeastSqPolyMod_gr_108.gif] coefficients  [Graphics:Images/LeastSqPolyMod_gr_109.gif]  so that the function  [Graphics:Images/LeastSqPolyMod_gr_110.gif]  given by the linear combination  
    
        [Graphics:Images/LeastSqPolyMod_gr_111.gif]  

will minimize the sum of the squares of the errors

        [Graphics:Images/LeastSqPolyMod_gr_112.gif].

Theorem (Linear Least Squares).  The solution to the linear least squares problem is found by creating the matrix  [Graphics:Images/LeastSqPolyMod_gr_113.gif]  whose elements are  [Graphics:Images/LeastSqPolyMod_gr_114.gif]

        [Graphics:Images/LeastSqPolyMod_gr_115.gif]
        
The coefficients [Graphics:Images/LeastSqPolyMod_gr_116.gif] are found by solving the linear system

        [Graphics:Images/LeastSqPolyMod_gr_117.gif]

where  [Graphics:Images/LeastSqPolyMod_gr_118.gif] and  [Graphics:Images/LeastSqPolyMod_gr_119.gif].

Proof  Least Squares Polynomials  Least Squares Polynomials  

 

Example 8.  Use the linear least squares method to find the polynomial curve fit of degree = 3  for the points [Graphics:Images/LeastSqPolyMod_gr_120.gif].
Solution 8.

 

Old Lab Project (Least Squares Polynomials  Least Squares Polynomials).  Internet hyperlinks to an old lab project.  

 

Research Experience for Undergraduates

Least Squares Polynomials  Least Squares Polynomials  Internet hyperlinks to web sites and a bibliography of articles.  

 

Download this Mathematica Notebook Least Squares Poynomials

 

Return to Numerical Methods - Numerical Analysis

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

(c) John H. Mathews 2004