Images as Tensors

Make that concrete. Here is a tiny $3 \times 3$ grayscale image written out as the numbers a model actually receives, 0 is black, 255 is white, and values between are shades of gray:

 12    34   200
 45   178   255
  9    87   120

Nine pixels, nine numbers, nothing more. A real $224 \times 224$ photo is the same object at scale, and a color version simply stacks three such grids, one for red, one for green, one for blue.

A color image adds a channel axis. An RGB image stores three numbers per pixel, one each for the red, green, and blue components, so it is a tensor of shape $H \times W \times 3$. The three channels are three stacked grayscale images that combine to produce color. Other channel counts exist: a single channel for grayscale or depth maps, four for RGBA (adding an alpha/transparency channel), and many more for satellite or medical imagery.

Bit depth: 8-bit integers versus normalized floats

A pixel's value needs a numeric type. The near-universal capture format is unsigned 8-bit integer (uint8): each channel value is a whole number in $[0, 255]$, giving $2^8 = 256$ distinct intensity levels per channel. That is why an RGB color is often written as three numbers from 0 to 255, and why $256^3 \approx 16.7$ million colors is the familiar figure.

Neural networks, however, do not want raw integers in $[0, 255]$. Large-magnitude inputs make gradients unstable and slow to converge, so the first step of nearly every vision pipeline is to convert to floating point and rescale into a small range, typically $[0, 1]$ by dividing by 255, and then standardizing further (below). The image passes from uint8 capture format to float32 model-input format.

Why we normalize

Rescaling to $[0, 1]$ is only half the story. The stronger, standard practice is standardization: subtract a per-channel mean and divide by a per-channel standard deviation so each channel is centered near zero with roughly unit variance. Models pretrained on ImageNet use fixed statistics computed once over that dataset, mean $(0.485, 0.456, 0.406)$ and standard deviation $(0.229, 0.224, 0.225)$ for the red, green, and blue channels. The full transform for one channel value $x$ in $[0, 255]$ is:

Take a mid-gray red value $x = 128$. Dividing by 255 gives $0.502$; subtracting the red mean $0.485$ and dividing by the red standard deviation $0.229$ gives $x_{\text{norm}} = (0.502 - 0.485)/0.229 \approx 0.074$. The pixel is now a small, centered number the optimizer can work with.

Worked example: how big is one image?

A standard model input is a $224 \times 224$ RGB image. Its element count is:

Memory depends entirely on the data type. In uint8, each value is 1 byte, so the image is $150{,}528$ bytes, exactly $147$ KiB. Converted to float32, each value takes 4 bytes:

The float version costs 4x the memory of the integer version for identical pixels, which is why images are stored and moved as uint8 and cast to float only inside the pipeline. It also compounds fast: a training batch of 32 such images in float32 is about $32 \times 588 \approx 18.4$ MiB before a single feature map is computed.

Memory layout: row-major, channel-first vs channel-last

The tensor is conceptually multi-dimensional, but RAM is a flat one-dimensional line of bytes. Row-major order (the C and NumPy default) stores the last axis contiguously: consecutive bytes walk across a row before stepping down to the next. The framework tracks a stride per axis to translate an index like (row, col, channel) into a flat offset.

Two conventions decide where the channel axis sits, and mixing them up is a frequent source of shape errors:

The leading $N$ is the batch dimension: models process many images at once, so a real input tensor is 4D, $N \times C \times H \times W$ in PyTorch, $N \times H \times W \times C$ in TensorFlow. Converting between the two layouts is a cheap axis permutation (transpose/permute), but forgetting to do it feeds a channel-first tensor to a channel-last op and produces either a crash or, worse, a model that trains on scrambled data.

Further reading

• CS231n: Neural Networks, data preprocessing, why centering and scaling inputs matters, with the standardization math.
• torchvision.transforms.Normalize, the standard normalization transform and the ImageNet statistics it is used with.
• Pillow: Concepts, modes and bands, how image modes (L, RGB, RGBA) map to channel counts.
• Dive into Deep Learning: Computer Vision, image tensors, layout, and preprocessing in runnable code.