Linear & Polynomial Regression

See Also: Multiple Linear Regression  Nonlinear Regression  Data Table 

Polymath can fit a polynomial of degree n with the general form: 

        P(x) = a₀ + a₁*x + a₂*x^2 + . . . + a_n*x^n

or a linear equation when the equivalent first degree polynomial is specified

        P(x) = a₀ + a₁*x

where a₀, a₁, ..., a_n are regression parameters to a set of N tabulated values of x (a single independent variable) versus y (a single dependent variable). The highest degree allowed for a polynomial is N - 1 (thus n >= N - 1). The program calculates the coefficients a₀, a₁, ..., a_n by minimizing the sum of squares of the deviations between the calculated y or P(x) and the data for y.

The Data Table window for Example 3 is shown below where the tabs for 'Regression' and for 'Linear & Polynomial' have been selected.

The options available in this window are the following:

Dependent Variable: 
Select the dependent variable for regression.

Independent Variable: 
Select the independent variable for regression (indicated by 'x' in the regression equation above).

Polynomial Degree:
Select the degree of the polynomial (indicated by 'n' in equation above), select the '1 Linear' polynomial for linear regression.

Through origin: 
If this option is marked, the free parameter is set to zero in the regression model (a0=0).

Polynomial Integration:
If this parameter is marked, the integral of the polynomial between the specified boundaries is calculated.

Polynomial Differentiation:
If this parameter is marked, the derivative of the polynomial at the specified point is calculated.

  Export to Excel:
Click on this icon to automatically transfer this problem to Excel and to carry out the regression and the requested calculations.

Solve (pink arrow):
Click on this icon to carry out the regression and the requested calculations. 

Graph: 
If this option is marked, a graph showing the calculated curve (or points) and the data points is prepared and displayed.

Residuals: 
If this option is marked, a graph showing the deviation between the data and the calculated values of the dependent variable (error, residuals) points is prepared and displayed. 

Store Model in Column: 
Store the regression model and the calculated parameters in the next available empty column.

Report: 
If this option is marked, a report showing the regression model the numerical values and confidence intervals of the parameters and some additional statistical and other information are presented and displayed.


Linear and Polynomial Regression

To carry out a linear or polynomial regression, select the column name of the independent variable and the column name of the dependent variable and the order of the polynomial you wish to fit. If you know that the physical model represented by the data requires that a straight line or the polynomial should pass through the origin (i.e., if the independent variable's value is 0.0, then the dependent variable must also be 0.0), mark the "Through origin" cell. To obtain a plot of the calculated polynomial and the data points versus the independent variable, mark the "Graph" option. To obtain a residual plot, mark the "Residuals" option. 

Note that you are allowed to mark several polynomial orders at once. In such case several reports, graphs etc. will be generated for the various order polynomials. For selecting the polynomial, which represents the data best see, the section of "Assessing the quality of regression models".

Example 1: Fitting Polynomial of Degree 1 - Heat Capacity Data of Solid Hydrogen Bromide.

Consider the data set shown below that is from Example 3 - "Heat capacity" in the Polymath REG Regression Program.   The data set may be obtained within the Polymath REG program by clicking on the Examples button and holding until Example 3 : Heat capacity is highlighted.  This should bring the data into the Polymath Data Table.

Then verify that the "Regression" tab and the "Linear & Regression" tab are selected.  Indicate the Dependent Variable: Cp, Independent Variable: T, Polynomial Degree: 1 Linear, and check only Report. Press the purple  arrow to achieve a Polymath Report of the Polynomial Degrees (Linear Regression) solution that is shown below:

Model: Cp = a0 + a1*T

General
Regression including a free parameter
Number of observations = 18

Statistics

Source data points and calculated data points

Example 2: Fitting Polynomial of Degree 2 (with Graph and Residuals) - Heat Capacity Data of Solid Hydrogen Bromide.

Consider the data set from Example 3 - "Heat capacity" in the Polymath REG Regression Program.   The data set may be obtained within the Polymath REG program by clicking on the Examples button and holding until Example 3 : Heat capacity is highlighted.  This should bring the data into the Polymath Data Table. For this example, please click on the Regression tab at the bottom of the Data Table and select Dependent Variable: Cp, Independent Variable: T, Polynomial Degree: 2, and check Graph, Residuals, and Report.  The solution will result in three separate windows that can be easily observed if the Windows button is clicked and Cascade selected. The front window given the Polymath Regression Report for the 2nd degree polynomial as shown below:

Polymath Regression Graph

The Polymath Regression Graph window output, given below, presents a scaled plot of the 2^nd degree polynomial as a solid curve and the actual data points that were used in the filling of the polynomial as the circles.  The allows you to visually determine the goodness of fit for the polynomial.

Polymath Regression Residuals Plot

The Polymath Regression Residuals Plot, given below, presents a scaled plot of the difference between the dependent variable, Cp in this case, for each of the input data points for the 2^nd degree polynomial.  It is desirable for the residuals to appear to be randomly placed about the zero line. This allows you to visually determine the goodness of fit for the polynomial.

Example 3: Fitting Polynomial of Degree 3 (with Integration and Differentiation) - Heat Capacity Data of Solid Hydrogen Bromide.

Consider the data set from Example 3 - "Heat capacity" in the Polymath REG Regression Program.   The data set may be obtained within the Polymath REG program by clicking on the Examples button and holding until Example 3 : Heat capacity is highlighted.  This should bring the data into the Polymath Data Table. A Polymath solution as indicated in the Input window shown below.  Note that the "Regression" and "Linear & Polynomial" tabs have been pressed.

For this Example, please select the following options:

"Report" gives the problem results.

"Store Model in column" stores the calculated values for the polynomial in the Data Table under the selected column heading C03.

"Dependent Variable" allows selection of Cp.

"Independent Variable" allows selection of T.

"Polynomial Degree" allows section of 3.

"Through origin" eliminates the constant from the regression of the polynomial.

"Polynomial Integration" integrates the dependent variable T from T1 = 120 to T2 = 200 using the 3^rd degree polynomial.

"Polynomial Derivative" calculates the derivation value when the independent variable T = 150 using the 3^rd degree polynomial.

The Polymath Report for this problem gives the integration and differentiation results as reproduced below:

Model: Cp = a1*T + a2*T^2 + a3*T^3

Analytical polynomial integration
Cp = 0.2314379*T -0.0017116*T^2 + 4.459E-06*T^3
Integ(Cp,T1,T2) = 0.115719 *T^2 -0.0005705*T^3 + 1.115E-06*T^4
T(1) = 120
T(2) = 200
Integ(Cp,T1,T2) = 936.3613

Analytical polynomial derivative
Cp = 0.2314379*T -0.0017116*T^2 + 4.459E-06*T^3
d(Cp)/d(T) = 0.2314379 -0.0034231*T + 1.338E-05*T^2
T = 150
d(Cp)/d(T) = 0.0189173

General
Degree of polynomial = 3
Regression not including a free parameter
Number of observations = 18

Statistics

Source data points and calculated data points

The Data Table has been updated with column C03 which contains the calculated Cp from the 3rd degree polynomial:

Simultaneous Polynomial Degrees

You can request treatment of several  polynomial degrees by holding down the "CNTR" key and at the same time clicking on the desired degrees with the left button of the mouse. You may also check the "Residual" and "Graph" options to obtain the desired plots for all selected degrees, and then press the pink arrow to solve. A report containing the numerical results for the selected polynomials will be obtained along with separate graphs and residual plots for each polynomial.

Additional Useful Information

See Assessing the Quality of Regression Models.