4. Convolutional Neural Networks

Chapter 4
Convolutional Neural Networks

Going beyond multi-layer perceptrons, this lecture introduces convolutional neural networks (CNNs) for image classification. We present the core mathematical building blocks and their justification. The second half of the lecture is devoted to the discussion of evaluating models, training variations, and popular CNN architectures.

Before introducing convolutional neural networks, let us consider building a hypothetical image classifier, i.e., a model that takes an image as input and produces a label for the image as output. We could, for example, try to classify images of iris flowers into the three categories, Setosa, Versicolour, and Virginica, like we did in the last lecture. However, unlike the classifier in that lecture, which used four manually measured features, here we want to use the raw image pixels as features directly. We can think about constructing our classifier from a multi-layer perceptron acting on a vectorized version of the image. An schematic is shown in Figure 33.

Hypothetical image classifier using a two-layer perceptron architecture
Figure 33: Hypothetical image classifier using a two-layer perceptron architecture.

Let us assume that the image is 224-by-224 pixels and has three colours. The vectorized input size is therefore,

\begin{align} n &= 224 \times 224 \times 3 \tag{141}\\ &= 150,528 \tag{142}\end{align}

For the sake of argument, let’s have a 1000-dimensional hidden state and ten output categories (rather than just three iris types). The parameters of the multi-layer perceptron parameters depicted in Figure 33 then will have sizes

\begin{align} A \in \reals^{1000 \times 150528}, \quad b \in \reals^{1000}, \quad C \in \reals^{10 \times 1000}, \quad d \in \reals^{10} \tag{143}\end{align}

giving a total of 150,539,010 parameters. Assuming that we use a standard 32-bit floating-point representation (i.e., 4 bytes per parameter), that amounts to 602,156,040 bytes (around 0.5GB) just to store the parameters!

Not only is that an enormous number of parameters to store, but training a model with so many parameters is computationally difficult and would require a huge amount of (labeled) data. Convolutional neural networks provide a much more efficient approach, and also address other issues to do with robustness as we will see later in the lecture.