Feedforward Network

A feedforward network is the simplest kind of neural network: information enters, passes through a sequence of computations, and produces an answer. Nothing loops back to revise an earlier layer during that same prediction, so the network’s data flow moves in one direction—forward.

How the layers compute
A feedforward network is built from an input layer, one or more hidden layers, and an output layer. Each layer transforms a vector of numbers using learned weights and biases, then usually applies an activation function such as ReLU:

Linear transformation: each unit combines values from the previous layer.
Activation: a nonlinear function lets stacked layers represent curved, conditional relationships rather than just one large straight-line calculation.
Output transformation: the final layer produces values suited to the task, such as a number, a set of class scores, or probabilities.

Learning through the network
During training, the network performs a forward pass to calculate predictions and a loss. Backpropagation then sends information about that error backward through the same layers, calculating how each weight should change. An optimizer such as Adam applies those changes. A network with several hidden fully connected layers is commonly called a multilayer perceptron (MLP); in PyTorch, it is typically assembled from nn.Linear layers and activations.

Why its structure matters
Feedforward networks are a foundational building block because they make the relationship between depth, activations, and gradients easy to see. Without nonlinear activations, any number of stacked linear layers collapses into one linear transformation. With too many poorly configured layers, gradients can fade before reaching early weights, and learning stalls; unstable initialization or an excessive learning rate can instead make loss explode. Normalization, careful initialization, and skip connections—as in a ResNet—help deeper feedforward paths remain trainable. Even transformer blocks contain feedforward sublayers: they independently transform each position after attention has mixed information across positions.

A feedforward network is a neural network in which information flows only from inputs through one or more hidden layers to outputs, with no feedback or recurrent connections. Each layer applies learned weighted transformations and nonlinear activations; a multilayer perceptron is a common form. This directed structure makes the forward pass and gradient-based training well defined, enabling the network to learn mappings from inputs to predictions.

Imagine an assembly line: a raw item enters at one end, passes through several stations, and leaves as a finished product. A feedforward network works in a similar one-way flow. Information enters as input — perhaps pixels from a photo or details from a form — then moves through a series of layers that gradually turn it into an answer.

For example, when identifying a cat in a photo, early layers may notice simple visual clues, while later layers combine those clues into a decision: “cat” or “not cat.” The key idea is that information only travels forward, from input to answer. It does not loop back or keep a running memory of earlier inputs.