Generalized Linear Models, the game

Beyond the straight line

Ordinary linear regression assumes a continuous outcome with normally distributed errors and constant variance. Real outcomes often break those rules: yes/no answers, counts of events, or positive, skewed amounts like insurance claims.

A generalized linear model (GLM) keeps the familiar linear predictor but adds two pieces so it can model these outcomes properly. Flip the cards to meet the three components.

Match the family

Choose the distribution (family) that fits the outcome:

Link function lab

The linear predictor η can be any number from −∞ to +∞. The inverse link turns it into a valid mean for the outcome: any number for Normal, a probability between 0 and 1 for Binomial, a non-negative count for Poisson.

Code it in Python

The article’s example predicts loan default (loan_status: 1 = default) on a Kaggle credit-risk dataset with a Binomial GLM, which is logistic regression.

import pandas as pd
import statsmodels.api as sm
import statsmodels.formula.api as smf
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, confusion_matrix

df = pd.read_csv("credit_risk_dataset.csv").dropna()
df = pd.get_dummies(df, columns=["person_home_ownership", "loan_intent",
                                 "loan_grade", "cb_person_default_on_file"],
                    drop_first=True, dtype=int)
train_df, test_df = train_test_split(df, test_size=0.2, random_state=42)

features = 
formula = "loan_status ~ " + " + ".join(features)
model = smf.glm(formula=formula, data=train_df, family=sm.families.Binomial()).fit()
print(model.summary())

test_df["pred_prob"] = model.predict(test_df)
test_df["pred"] = (test_df["pred_prob"] > 0.5).astype(int)

print("Accuracy:", accuracy_score(test_df["loan_status"], test_df["pred"]))
print(confusion_matrix(test_df["loan_status"], test_df["pred"]))

Defeat the GLM Boss 🐲

Seven questions, three hearts.

Keep levelling up

You learned why plain linear regression struggles with binary, count and skewed outcomes, the three components of a GLM, how to match an outcome to its family, what link functions do, and how to fit a logistic GLM in statsmodels.