Multilayer Perceptron, the game

One neuron, by hand

A neuron computes a weighted sum of its inputs plus a bias, then applies an activation: output = f(w₁x₁ + w₂x₂ + b). Here f is a simple step: output 1 if the sum is above 0. Tune the weights so the neuron behaves like a logic gate.

The XOR wall

A single neuron can only separate classes with one straight line. XOR’s two “1” corners sit diagonally opposite each other, so no single line can split them. This limitation stalled neural network research in 1969.

The fix: add a hidden layer. One hidden neuron can learn OR and another NAND; the output neuron ANDs them together, and that is exactly XOR. A multilayer perceptron (MLP) is input layer → one or more hidden layers → output layer.

Activations and training

Real networks use smooth activations so they can be trained with gradients: sigmoid (0 to 1), tanh (−1 to 1), ReLU (max(0, z), fast and the default in hidden layers) and softmax (turns the output layer into class probabilities).

Code it in Python

The article detects network intrusions (Kaggle network-intrusion data) with scikit-learn’s MLPClassifier.

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.neural_network import MLPClassifier
from sklearn.metrics import classification_report

data = pd.get_dummies(pd.read_csv("Train_data.csv"), columns=["protocol_type", "service", "flag"])
X, y = data.drop(columns="class"), data["class"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

scaler = StandardScaler().fit(X_train)            # neural nets need scaled inputs
X_train, X_test = scaler.transform(X_train), scaler.transform(X_test)

mlp = MLPClassifier(hidden_layer_sizes=(64, 32), activation="relu",
                    solver="adam", max_iter=300, random_state=42)
mlp.fit(X_train, y_train)
print(classification_report(y_test, mlp.predict(X_test)))

Defeat the Neural Boss 🐲

Six questions, three hearts.

Keep levelling up

You learned how a neuron computes a weighted sum and an activation, why a single neuron can only draw one straight line, how hidden layers solve XOR, how training works with backpropagation, and how to train an MLPClassifier.