1x1 Convolution

A 1×1 convolution looks almost too small to be useful: it examines one position at a time, without looking at neighbouring positions. Its real job is not to detect spatial patterns such as edges or corners, but to decide how the feature channels already present at that position should be combined.

How it works
If a layer receives a feature map with, say, 64 channels, a 1×1 convolution can produce 128 new channels. At every spatial location, it takes that location’s 64 values, multiplies them by learned weights, adds a bias, and emits 128 values. The same learned channel-mixing rule is reused at every location. In this sense, it is like a small fully connected layer applied independently to each pixel-sized feature vector.

Why a tiny kernel is powerful
A 1×1 kernel has no spatial reach: with stride 1, it preserves height and width and cannot directly combine information from adjacent locations. But it is extremely useful between spatial convolutions because it can:

• Mix channels, combining detectors from earlier layers into more meaningful features.
• Change width cheaply, reducing 256 channels to 64 before an expensive 3×3 convolution, then expanding them again afterward.
• Add nonlinear depth when followed by an activation, allowing richer transformations without changing resolution.
• Downsample when used with a stride greater than 1.

Where it appears in real networks
The ResNet bottleneck block uses a 1×1 convolution to compress channels, a 3×3 convolution to process spatial neighbourhoods, and another 1×1 convolution to restore channel width. This cuts parameter count and compute dramatically: a 3×3 convolution from 256 to 256 channels needs about 590,000 weights, while a 1×1 version needs about 65,000. Used carelessly, aggressive channel reduction creates a narrow information bottleneck and harms accuracy; used well, 1×1 layers make deep networks both expressive and affordable.