Decision Trees in R, the game
Twenty questions, but for data. Learn how trees pick their splits, grow one deeper and deeper on real flowers, and spot the moment it starts to overfit.
A flowchart that learns
A decision tree is a non-parametric model: it makes no assumption about the shape of the data. It splits the data with a series of yes/no questions (“is petal length ≤ 2.45?”). Each internal node is a question, each branch an answer, and each leaf a final prediction.
How splits are chosen
Information gain = entropy before the split − weighted entropy after. The tree tries every possible split and keeps the one with the biggest gain (or biggest drop in Gini).
| Algorithm | Split criterion | Notes |
|---|---|---|
| ID3 | Information gain | Categorical features only |
| C4.5 | Gain ratio | Handles numbers and missing values; prunes; turns trees into if-then rules |
| C5.0 | Gain ratio | Faster, smaller trees, boosting |
| CART | Gini (classification) | Binary splits; also does regression; used by rpart and scikit-learn |
A node holds 6 “yes” and 2 “no”. What is its Gini impurity?
Grow the tree
Allow the tree to grow deeper and watch: training accuracy keeps climbing, but does test accuracy follow?
The playground uses R’s built-in iris data (150 flowers, 3 species) with just two features, sepal length and width, so you can see the decision regions. Every third flower is held out as a test set.
- ✓After exploring, go back to depth 1 (a “stump”)+20 XP
- ✓Find the depth with the best test accuracy+30 XP
- ✓Grow to depth 8 and compare training vs test+20 XP
Code it in R
The article predicts 10-year coronary heart disease (TenYearCHD) from the Framingham data with rpart. This version fixes the variable names (data vs df) and a stray quote.
Step 1Load and split
library(rpart)
library(rpart.plot)
library(caTools)
data <- read.csv("CHD_preprocessed.csv")
dim(data); str(data)
set.seed(123)
split <- sample.split(data$TenYearCHD, SplitRatio = 0.75)
training_set <- subset(data, split == TRUE)
test_set <- subset(data, split == FALSE)Step 2Grow (and limit) the tree
tree <- rpart(TenYearCHD ~ ., data = training_set, method = "class",
control = rpart.control(maxdepth = 4, cp = 0.01))
rpart.plot(tree)maxdepth and the complexity parameter cp stop the tree from overfitting.
Step 3Evaluate
prediction <- predict(tree, newdata = test_set, type = "class")
cm <- table(test_set$TenYearCHD, prediction)
accuracy <- sum(diag(cm)) / sum(cm)
print(paste("Accuracy on test data:", round(accuracy, 3)))Only about 15% of patients in this data develop CHD, so always look at the confusion matrix, not just accuracy.
Which rpart() argument tells it to build a classification tree?
Defeat the Tree Boss 🐲
Seven questions, three hearts.
You’re a Tree Master!
Keep levelling up
You learned how a decision tree asks yes/no questions, how entropy, information gain and Gini choose the splits, the ID3/C4.5/CART family, why deep trees overfit, and how to fit one with rpart.