Lasso Regression in R, the game

Meet the villain: overfitting

A model overfits when it learns the training data too well, noise included. It scores brilliantly on data it has seen and badly on data it hasn’t, because it has become too complex: too many features, or weights that are too large.

Regularization fights this. It adds a small penalty for complexity, so the model prefers simpler explanations that generalize to new data. Lasso is one of the most popular regularization techniques.

Decode the name

Lasso is a special type of linear regression that adds a penalty based on the absolute values of the coefficients (the numbers in front of each feature). That penalty shrinks unimportant coefficients, and can push some of them all the way to zero, removing those features from the model. This is called L1 regularization.

Lasso is especially useful when:

• your dataset has many features, maybe more than you have rows;
• some features are very similar to each other (multicollinearity);
• you want the model to pick the useful features for you.

Pull the λ lever

Lasso minimizes the usual error plus a penalty on the size of the coefficients:

• λ = 0: no penalty, so you get ordinary linear regression with every feature.
• Bigger λ: coefficients shrink and unimportant ones hit exactly zero. More bias, less variance.
• Very large λ: every coefficient is zero and the model just predicts the average.

The playground uses the same algorithm as glmnet (coordinate descent on standardized features, 100 λ values) on R’s built-in mtcars data. At the smallest λ the coefficients match ordinary least squares, exactly as the theory says.

Lasso vs Ridge: rapid fire

Ridge regression also adds a penalty, so what’s the difference? Lasso uses L1 (absolute values, |β|) and Ridge uses L2 (squares, β²). The L1 penalty produces sparse models: some coefficients become exactly zero. Ridge shrinks every coefficient towards zero but keeps them all.

Code it in R

Time to run Lasso yourself with the glmnet package. The workflow: load the data, split features and target, fit the model, then use cross-validation to pick the best λ.

install.packages("glmnet")
library(glmnet)

data(mtcars)
head(mtcars)

target   <- mtcars$mpg
features <- as.matrix(mtcars[, -which(names(mtcars) == "mpg")])

lasso <- glmnet(x = features, y = target, alpha = 1)

# How each coefficient shrinks as λ grows
plot(lasso, xvar = "lambda", label = TRUE)

set.seed(42)  # CV folds are random; this makes results repeatable
cv_lasso <- cv.glmnet(x = features, y = target, alpha = 1)

# Cross-validation error for every λ
plot(cv_lasso)

# Best lambda value
best_lambda <- cv_lasso$lambda.min
print(paste("Best lambda:", best_lambda))

# Coefficients at the best λ (a "." means the feature was dropped)
coef(cv_lasso, s = "lambda.min")

Strength or weakness?

Every algorithm has trade-offs. Lasso is great at simplifying models, but it isn’t magic. Sort each statement to see where it shines and where it struggles.

Defeat the Overfit Dragon 🐉

Seven questions and three hearts. A wrong answer costs a heart; lose all three and the dragon wins this round (you can try again). Beat it to earn the Lasso Master badge.

Keep levelling up

In this lesson you learned what Lasso regression is, how the λ penalty works, how it differs from Ridge, how to run it in R with glmnet, and where it shines or struggles.