Introduction to Regression, the game
Start your regression journey by playing. Five quick missions on what regression predicts, bias and variance, residuals and the whole regression family, then a boss quiz.
What does regression predict?
Regression is a supervised machine learning technique that learns the relationship between independent variables (the inputs, also called features) and a dependent variable (the outcome). Its job is to predict a continuous number.
Example: predicting a house’s price from the number of rooms, the size of the land and the distance to the airport. The price is the dependent variable; rooms, land size and distance are the independent variables.
If the outcome is a category instead (spam or not spam, which species), that is classification, not regression.
In “predict a house’s price from rooms, land size and distance to the airport”, which is the dependent variable?
Bias, variance and the dartboard
Bias is how far the model’s predictions are from the true values on average: a systematic miss. Variance is how much the predictions would change if you trained on a different sample of data: how jumpy the model is.
- Underfitting: the model is too simple and does badly on training and test data. Usually high bias.
- Overfitting: the model memorizes the training data and does badly on new data. Usually high variance.
- Bias–variance trade-off: making a model more flexible lowers bias but raises variance. The goal is the sweet spot with the lowest total error.
Think of each dot below as one model trained on a different sample. The bullseye is the true value.
Which dartboard shows high bias, low variance (a consistent but wrong model)?
A model scores 99% on its training data but only 60% on test data. What is happening?
Coefficients and residuals
Regression coefficients tell you how the outcome changes with each feature. Their sign gives the direction of the relationship:
- Positive: the outcome rises as the feature rises (salary with years of experience).
- Negative: the outcome falls as the feature rises (a car’s fuel efficiency as its weight increases).
A residual is the vertical distance between a data point and the regression line: residual = actual − predicted.
| Friend | Real age (y) | Your guess (ŷ) | Residual (y − ŷ) |
|---|---|---|---|
| Amit | 10 | 8 | +2 |
| Riya | 15 | 12 | +3 |
| Rahul | 20 | 17 | +3 |
A good residual plot looks like random noise scattered evenly around zero, with no pattern.
Neha is 25 and you guessed 28. What is her residual?
Meet the regression family
There isn’t one regression model; there is a whole family. Flip each card to meet them.
Defeat the Regression Boss 🐲
Seven questions, three hearts. Lose all three and the boss wins this round (you can try again).
You’re a Regression Rookie Champion!
Keep levelling up
You learned what regression predicts, the difference between dependent and independent variables, bias, variance, residuals, and when to reach for each type of regression.