Polynomial Regression in R, the game

When a line isn’t enough

Linear regression fits a straight line, so it does poorly on curved data. Polynomial regression adds powers of the feature (x², x³…) so the model can bend:

It is still called linear regression because the model is linear in its coefficients; only the features are curved.

Turn the degree dial

A higher degree makes the curve more flexible. That lowers training error every time, but the error on new data tells the real story.

Below: real mtcars data, mpg against horsepower. Black cars are used for training and hollow ones are held out as a test set.

Code it in R

The article uses R’s built-in pressure dataset (vapour pressure of mercury against temperature). This version fixes three issues in the original: it uses all 19 rows (not just head()), trains on the training set only, and predicts on the test set.

library(caTools)
library(Metrics)
library(ggplot2)

data <- pressure                       # all 19 rows
set.seed(123)
split    <- sample.split(data$pressure, SplitRatio = 2/3)
training <- subset(data, split == TRUE)
testing  <- subset(data, split == FALSE)

# Degree-4 polynomial in temperature
poly_reg <- lm(pressure ~ poly(temperature, 4, raw = TRUE), data = training)
summary(poly_reg)

pred <- predict(poly_reg, newdata = testing)
rmse(testing$pressure, pred)
mae(testing$pressure, pred)

grid <- data.frame(temperature = seq(0, 360, 1))
ggplot() +
  geom_point(aes(temperature, pressure), data = data, colour = "red") +
  geom_line(aes(grid$temperature, predict(poly_reg, newdata = grid)), colour = "blue") +
  ggtitle("Polynomial regression (degree 4)")

Strength or weakness?

Sort each statement about polynomial regression.

Defeat the Curve Boss 🐲

Six questions, three hearts.

Keep levelling up

You learned when polynomial regression beats a straight line, how the degree controls the bias–variance trade-off, how to spot overfitting with a test set, and how to fit it in R.