Artificial Neuron

An artificial neuron is a small calculation unit that turns several input numbers into one output number. By itself, it is simple; thousands or billions of these units connected in layers can form a network that learns useful patterns from data.

What a neuron computes
Each input is multiplied by a trainable weight, and the results are added together with a trainable bias: z = w₁x₁ + w₂x₂ + ... + b. The value z is the pre-activation. The neuron then applies an activation function, such as ReLU, producing a = ReLU(z). Weights determine which inputs matter and in which direction; the bias shifts the point at which the neuron becomes active. A ReLU neuron, for example, outputs zero for negative pre-activations and passes positive values onward.

Why activations are essential
Without activation functions, stacking many layers would still collapse into one large linear calculation. Nonlinear activations let combinations of neurons represent curved boundaries, conditional rules, and progressively more abstract features. A layer does this calculation for many neurons at once: its input is a vector, its weights form a matrix, and its output becomes the next layer’s input. In PyTorch, nn.Linear provides the weighted-sum-and-bias part, while nn.ReLU supplies the activation.

How neurons learn—and fail
During training, backpropagation calculates how changing every weight and bias would change the loss. An optimiser such as Adam then adjusts them. A neuron that repeatedly receives useful gradient signals strengthens or weakens its connections to help the network’s prediction.

Vanishing gradients: early neurons receive nearly zero updates, so deep layers stop learning.
Dying ReLUs: a neuron can produce zero for every training example and receive no useful gradient through ReLU.
Exploding activations or gradients: values grow rapidly across layers, making loss curves unstable or divergent.

Careful initialization, suitable activations, normalization, and residual connections such as those in ResNet help keep these tiny calculations trainable at depth.

An artificial neuron is a computational unit that multiplies input values by learned weights, adds a learned bias, and applies an activation function to produce an output. Connected neurons form layers that learn increasingly useful representations. It matters because adjusting each neuron’s weights and bias through gradient-based training enables the network to model complex, non-linear relationships.

Think of an artificial neuron as a tiny decision-maker in a large team. It receives several pieces of information—perhaps pixels from a photo or words from a sentence—and decides how strongly to pass along a particular signal. For example, one neuron might become especially responsive to “this looks like an edge,” while another notices “this sounds positive.”

By itself, a neuron is very limited. But when many are connected in layers, they can combine simple clues into richer ideas: edges into faces, or letters into meaning. During training, the network adjusts each neuron’s connections based on examples, gradually improving the team’s decisions.