Polynomial Regression
Polynomial Regression
less than a minute
What if our data is more complex than a straight line?
We can use a linear model to fit non-linear data.
Add powers of each feature as new features, then train a linear model on this extended set of features.
Polynomial Regression
Linear: \(\hat{y_i} = w_0 + w_1x_{i_1} \)
Polynomial (quadratic): \(\hat{y_i} = w_0 + w_1x_{i_1} + w_2x_{i_1}^2\)
Polynomial (n-degree): \(\hat{y_i} = w_0 + w_1x_{i_1} + w_2x_{i_1}^2 +w_3x_{i_1}^3 + \dots + w_nx_{i_1}^n \)
Above polynomial can be re-written as linear equation:
\[\hat{y_i} = w_0 + w_1X_1 + w_2X_2 +w_3X_3 + \dots + w_nX_n \]where, \(X_1 = x_{i_1}, X_2 = x_{i_1}^2, X_3 = x_{i_1}^3, \dots, X_n = x_{i_1}^n\)
=> the model is still linear w.r.t to its parameters/weights \(w_0, w_1, w_2, \dots , w_n \).
e.g:
Strategy to find Polynomial Features
- Fit a linear model to the data points.
- Plot the errors.
- If the variance of errors is too high, then try polynomial features.
Note: Detect and remove outliers from error distribution.
High Degree Polynomial Regression
Conclusion
- Polynomial model : Over-fitting ❌
- Linear model : Under-fitting ❌
- Quadratic model: Generalizes best ✅
End of Section