Convolution Operation

A convolution operation lets a network inspect a small neighbourhood at a time rather than treating every input value as unrelated. A compact set of learned numbers, called a kernel or filter, slides across a grid and asks the same learned question at every location: “Does this local pattern appear here?”

What the operation computes
For each position, the layer multiplies corresponding input values and kernel weights, adds the results and a bias, and writes one number into an output feature map. With a colour image, for example, a kernel spans all input channels: a 3×3 kernel on RGB data contains 3×3×3 learnable weights. Each output channel has its own kernel, so different filters can respond to different patterns. Deep-learning libraries usually implement cross-correlation—sliding the kernel without flipping it first—though they call the layer “convolution.” Since the weights are learned, that naming difference has no practical consequence.

Why sliding and sharing matter
The same kernel weights are reused at every location. This weight sharing gives convolution two useful properties:

• Local connectivity: a unit sees a small receptive field, capturing nearby structure efficiently.
• Translation equivariance: shifting a pattern in the input shifts its response in the feature map.
• Far fewer parameters: a 3×3 kernel remains 3×3 regardless of grid width and height, unlike a fully connected layer.

Training controls and practical consequences
Stride sets how far the kernel moves each step; larger strides shrink the output but discard spatial detail. Padding adds border values, commonly zeros, so edge information is not immediately lost. Stacking convolutions expands the effective receptive field: later layers combine evidence from wider regions. In PyTorch, torch.nn.Conv2d packages these choices into one trainable layer. Poor choices can damage learning: excessive stride or no padding can erase useful boundary and fine-grained signals, while very large kernels raise compute and parameter cost. Gradients flow through every multiply-and-add, allowing each filter to become a useful detector rather than a hand-designed one.