Convolution & CNNs

Convolution makes two structural bets instead:

• Local connectivity, each output looks at only a small neighborhood of the input, because visual features (edges, corners, textures) are local.
• Weight sharing, the same small set of weights is reused at every position, because a useful feature detector is useful everywhere in the image.

The convolution operation: kernel, stride, padding

A kernel (or filter) is a small weight grid, typically $3 \times 3$ or $5 \times 5$, spanning all input channels. You slide it across the image; at each position you multiply the kernel elementwise with the patch beneath it and sum the products into a single output number. Sweep over every position and the outputs form a feature map, a new grid that lights up where the kernel's pattern is present.

Three hyperparameters control the sweep:

• Kernel size $K$, the side length of the patch each output sees.
• Stride $S$, how many pixels the kernel jumps between positions. Stride 2 skips every other position, halving the output resolution.
• Padding $P$, rings of (usually zero) pixels added around the border, so the kernel can be centered on edge pixels and the output does not shrink.

The output side length follows directly:

Two cases worth memorizing, both starting from $W = 224$:

• $K=3,\ P=1,\ S=1$: $\lfloor (224 - 3 + 2)/1 \rfloor + 1 = 224$. A $3 \times 3$ kernel with padding 1 and stride 1 preserves the resolution, the ubiquitous "same" convolution.
• $K=3,\ P=0,\ S=2$: $\lfloor (224 - 3)/2 \rfloor + 1 = \lfloor 110.5 \rfloor + 1 = 111$. Stride 2 with no padding roughly halves the resolution, downsampling as it goes.

Weight sharing and local connectivity

The payoff shows up in the parameter count. A convolutional layer stores one small weight tensor per output channel, of shape $K \times K \times C_{\text{in}}$, plus one bias, regardless of image size. For 64 filters of size $3 \times 3$ over 3 input channels:

That $1{,}792$-parameter layer produces a $224 \times 224 \times 64$ feature map, a far richer output than the dense layer's, at roughly one hundred-thousandth of the weights. And because the same kernel is applied everywhere, the layer is translation-equivariant: shift the input and the feature map shifts with it, instead of becoming unrecognizable.

Pooling and the receptive field

Stacked convolutions alone keep the resolution high and expensive. Pooling downsamples a feature map by summarizing each small window into one value, max pooling takes the largest activation in each $2 \times 2$ window (keeping the strongest evidence of a feature), average pooling takes the mean. A $2 \times 2$ pool with stride 2 halves both spatial dimensions, so $224 \times 224 \to 112 \times 112$, cutting the activation count by 4x and granting a little tolerance to small shifts.

The receptive field is the region of the original image that influences one output value. A single $3 \times 3$ conv has a $3 \times 3$ receptive field; stack a second and it grows to $5 \times 5$; a third reaches $7 \times 7$. Pooling and strided convolution expand it much faster. This is the engine behind the classic hierarchy: early layers, with tiny receptive fields, detect edges and colors; deeper layers, seeing large swaths of the image, respond to textures, then object parts, then whole objects.

This is also why two stacked $3 \times 3$ convolutions are preferred over one $5 \times 5$: they cover the same receptive field with fewer parameters and an extra nonlinearity in between.

A typical conv–pool stack

Classic CNNs (LeNet, AlexNet, VGG) repeat a simple motif: a few convolutions with ReLU nonlinearities, then a pool that halves resolution, with the channel count growing as the spatial size shrinks, trading spatial detail for feature richness.

1. 1Conv block 1Two 3x3 convolutions (padding 1, stride 1) with ReLU keep resolution at 224 and lift channels to 64, learning edges and simple color patterns.
2. 2Pool 1A 2x2 max pool with stride 2 halves resolution to 112, discarding precise position while keeping the strongest activations.
3. 3Conv block 2More 3x3 convolutions raise channels to 128 at 112 resolution, combining edges into textures and corners over a larger receptive field.
4. 4Pool and deepenRepeat: each stage halves the spatial size and doubles the channels, so the receptive field widens and features grow from parts toward whole objects.
5. 5HeadGlobal pooling or a flatten feeds a small fully-connected classifier that maps the final feature vector to class scores.

Further reading

• CS231n: Convolutional Neural Networks, kernels, stride, padding, pooling, and receptive fields with clear diagrams.
• A guide to convolution arithmetic for deep learning (Dumoulin & Visin, 2016), the definitive reference on output sizes, padding, and strides.
• Deep Learning, Chapter 9: Convolutional Networks (Goodfellow, Bengio, Courville), the theory of local connectivity, weight sharing, and equivariance.
• Dive into Deep Learning: Convolutional Neural Networks, the operation and a full CNN built in runnable code.