Multilayer Perceptron (MLP)

An Multilayer Perceptron (MLP) is the classic “stack of neural-network layers”: values enter at one end, are repeatedly transformed by learned connections and nonlinear activations, and leave as a prediction. Despite its simple shape, an MLP is a powerful general-purpose function learner.

How an MLP computes

An MLP has an input layer, one or more hidden layers, and an output layer. In its usual form, every unit in one layer connects to every unit in the next, making these fully connected (or dense) layers. Each layer computes a weighted sum plus a bias, then applies an activation such as ReLU or GELU:

h = activation(Wx + b)

Without activations, several stacked dense layers collapse mathematically into one linear transformation. Nonlinear activations are what let depth represent curved decision boundaries and complicated relationships.

How it learns

During the forward pass, an MLP produces predictions and a loss measures their error. Backpropagation sends information about that error backward through every layer, producing gradients for each weight. An optimizer such as Adam then adjusts the weights to reduce future error. Deeper MLPs need careful design because gradients can fade or explode while crossing many layers. ReLU-family activations, suitable initialization, LayerNorm, and residual connections help keep the signal and gradients in a usable range.

Practical trade-offs

MLPs work naturally when inputs are already fixed-length feature vectors: tabular measurements, learned embeddings, or the final classifier head of another network. A transformer, for example, commonly uses an MLP—called a feed-forward block—after attention. Their main cost is parameter growth: connecting a layer of 4,000 units to another 4,000-unit layer requires 16 million weights. Excessively wide MLPs consume memory and can overfit; dropout disables random hidden units during training to discourage reliance on any single path, then remains off during inference. A loss curve that falls briefly and then diverges usually points to an overly large learning rate, unstable activations, or poorly scaled inputs—not a mysterious failure of the MLP itself.

A multilayer perceptron (MLP) is a feed-forward neural network consisting of an input layer, one or more hidden layers of fully connected neurons with nonlinear activations, and an output layer. Each layer transforms the preceding layer’s outputs through learned weights and biases. MLPs can model complex nonlinear functions; without hidden layers and nonlinearities, the network reduces to a single linear transformation.

Imagine a team passing a clue along a line. Each person notices something different and adds their own judgment before handing it on. By the end, the team can make a useful decision from many small observations.

A Multilayer Perceptron (MLP) is a simple kind of AI network built in a similar way. Information enters at one end, passes through several groups of connected “decision-makers,” and comes out as an answer: perhaps “this email looks like spam” or “this customer may like this product.”

The extra layers matter because they let the network learn increasingly useful patterns from examples, rather than relying on a person to spell out every rule.