Curve fitting is nothing but approximating the given function f(x) using simpler functions say polynomials, trignometric functions, exponential functions and rational functions. However, the main difference between interpolation and Curve fitting is, in the former, the approximated curve has to pass through the given data points. Here again polynomial functions are the one which are been used widely in the applications than the other functions. The existence of a polynomial function P(x) which approximate any continuous function f(x) as a finite interval [a, b] is gauranteed by Weiestrass approximation theorem.

Weiestrass approximation Theorem : If the function f(x) is continuous as a finite interval [a, b], then given any e > 0 then exists a n = n(e) and a polynomial P(x) of degree n such that 
| f(x) - P(x) | < e for all x Î[a, b].

In curve fitting f(x) will be taken as

f(x) @  P(x, c₀, c₁ . . . c_n)
 = c₀F₀(x) + c₁F₁(x) +  . . . + c_nF_n(x)

where F_i(x), i = 0, 1, . . . n are n chosen linearly independent functions(xⁱfor polynomials) and 
c_i, i = 0, 1, 2, . . . n are parameters to be determined.

The error in the approximation is defined as

E(f;c) = || f(x) - (c₀F₀ + . . . + c_nF_n) ||

where || || is a norm. Now the problem is to find c_i, i = 0, 1, 2, . . . n such that E(f;c) is as small as possible in the sense of the norm || ||.

The most commonly used norms :

For discrete data L^p - norm defined as

where x = {x_i} is a sequence of real or complex numbers is most commonly used norms.
The special cases in L^p - norm are for p = 2 and p = µ  i.e., respectively the Euchlidean norm 

out of these norms, Euchlidean norm is most widely used which results in least squares curve fitting. Now the error function in least square sence is defined as

where W( x_i ) is some weighting function.

Then minimization of E(f; c_i) with F_i = xⁱ, i = 0, 1, . . . n and W( x_i ) = 1 gives the condition

¶E / ¶c_i = 0  for i = 0, 1, . . . n

called normal equations. This gives a system of (n + 1) linear equations for (n + 1) unknowns 
c_i ,   i = 0, 1, . . . n.

Normal equations :

Case(i)

If f(x) is approximated with a linear polynomial i.e., P₁(x) = c₀ + c₁x and W(x) =1 then

c₀n + c₁Sx_i = Sf_i    and  c₀Sx_i + c₁Sx_i² = +Sx_if_i

 
Case(ii)

If f(x) is approximated with a second degree polynomial i.e., P₂(x) = c₀ + c₁x+ c₂x²
with W(x) = 1. Then

Þc₀n + c₁Sx_i + c₂Sx_i² = Sf_i
    c₀Sx_i+ c₁Sx_i² + c₂Sx_i³ = +Sf_ix_i
    c₀Sx_i²+ c₁Sx_i³ + c₂Sx_i⁴ = +Sf_ix_i²

 
Case(iii)

If f(x) is approximated with a m^th degree polynomial 
i.e., P_m(x) = c₀ + c₁x+ . . . + c_mx^m    with  W(x) = 1.

then the normal equations are:

c₀n + c₁Sx_i + c₂Sx_i²  + . . . +c_mSx_i^m = Sf_i
c₀Sx_i+ c₁Sx_i² + c₂Sx_i³   + . . . +c_mSx_i^m+1 = +Sf_ix_i

.	.	.	.	.
.	.	.	.	.
.	.	.	.	.

c₀Sx_i^m+ c₁Sx_i^m+1 + c₂Sx_i^m+2   + . . . +c_mSx_i^2m = +Sf_ix_i^m

Example - 1 :

Find the least squares line for the data points

The normal equations for P(x) = c₀ + c₁x are   c₀n + c₁Sx_i = Sf_i      c₀Sx_i + c₁Sx_i² = +Sx_if_i
Þ 8c₀ + 20c₁ = 37
    20c₀ + 92c₁ = 25
Þ c₀= 8.643     c₁= -1.607

The required line P₁(x) is 8.643 - 1.607x
 

Example - 2 :

Find the least squares second degree curve for the data

let the curve be P(x) = c₀ + c₁x+ c₂x²
Normal equations are

c₀n + c₁Sx_i + c₂Sx_i² = Sf_i
c₀Sx_i+ c₁Sx_i² + c₂Sx_i³ = +Sf_ix_i
c₀Sx_i²+ c₁Sx_i³ + c₂Sx_i⁴ = +Sf_ix_i²

consider the transformation X = (x - 6) / 2

Now if we use above normal equations replacing x_iwith X_i we get
5c₀ + 0 + 10c₂ = 196.06

0 + 10c₁ + 0 = 220.97

10c₀ + 0 + 34c₂ = 447.67

c₀ = 31.276   c₁ = 22.097   c₂ = 3.9679

P(X) = 31.276 + 22.097X + 3.9679X²

(or)

The required line P₂(x) is  0.696 - 0.855x + 0.992x²
 

WORK OUT LEAST SQUARES CURVE HERE :