CNN in Practice: Flattening, Fully-Connected Layers, Softmax & Implementation

How a stack of feature maps becomes a class prediction — and how everything since Lecture 4 (dense layers, cross-entropy, backprop, Adam) reassembles inside a real, runnable Keras CNN.

• Explain why a nonlinearity is applied after every convolution, and that it doesn't change the feature map's shape.
• Describe flattening as a pure reshape with no learned parameters.
• Derive the softmax + categorical cross-entropy gradient and connect it explicitly to Lecture 8's sigmoid+BCE shortcut.
• Write the batch normalization equations and explain what γ and β are for.
• Explain how residual connections F(X)+X keep gradients flowing in very deep networks.
• Build, compile and train a complete Keras CNN for MNIST digit classification.

1. Nonlinearity After Convolution

Convolution itself — a sum of elementwise products — is a strictly linear operator. Stack linear operations on top of linear operations and the whole network collapses mathematically into a single linear map, no matter how many layers deep (exactly the argument from Lecture 4/5 for dense networks). So, exactly as with a dense layer, every convolution is followed by an elementwise nonlinearity — in practice almost always ReLU.

2. Flattening

After the last convolution+pooling block, the network holds a 3D feature map of shape \(H''\times W''\times D_K\) — height, width, and channel depth. To feed this into the dense classifier layers built in Lecture 4, it is flattened: reshaped into a single 1D vector of length \(H''\times W''\times D_K\).

3. Fully Connected Layers, Softmax & Cross-Entropy

The flattened feature vector passes through one or more dense layers — exactly the feedforward math of Lecture 4 and the backprop machinery of Lecture 8, unchanged. For multi-class classification, the final layer is a softmax, which turns a vector of raw scores (logits) \(z\) into a valid probability distribution over \(C\) classes:

$$\text{softmax}(z)_i=\frac{e^{z_i}}{\sum_{j=1}^{C} e^{z_j}}$$

paired with categorical cross-entropy loss against a one-hot target \(y\) (the direct multi-class generalization of Lecture 6's binary cross-entropy, derived there from maximum likelihood):

$$L = -\sum_{i=1}^{C} y_i\log(\hat y_i), \qquad \hat y=\text{softmax}(z)$$

Lecture 8 showed that a sigmoid output paired with binary cross-entropy collapses to the remarkably clean gradient \(\partial L/\partial Z_2 = Q-y\). The same thing happens here — but that's a claim, not yet a fact, until it's actually derived rather than asserted by analogy. It's worth doing in full, because it is one of the most-used shortcuts in all of deep learning.

You can replay the same four steps interactively below — useful for testing yourself before moving on:

A small worked example makes this concrete. Suppose a 3-class classifier produces logits \(z=[2.0,\ 1.0,\ 0.1]\) and the true class is class 0 (\(y=[1,0,0]\)):

The negative first component says "increase logit 0" (push the correct class's score up); the positive remaining components say "decrease logits 1 and 2" (push the wrong classes' scores down) — exactly the direction that would reduce the loss on the next gradient-descent step (Lecture 7).

4. Batch Normalization

As networks get deep (recall ResNet's 152 layers, Lecture 12), the distribution of each layer's inputs keeps shifting as earlier layers' weights update during training — a problem often called internal covariate shift. Batch normalization re-centers and re-scales each layer's inputs using statistics computed over the current mini-batch:

$$\mu_B=\frac1{N_B}\sum_i x_i \qquad \sigma_B^2=\frac1{N_B}\sum_i(x_i-\mu_B)^2$$ $$\hat x_i=\frac{x_i-\mu_B}{\sqrt{\sigma_B^2+\epsilon}} \qquad y_i=\gamma\hat x_i+\beta$$

Here \(N_B\) is the batch size and \(\epsilon\) a tiny constant preventing division by zero. \(\gamma\) and \(\beta\) are trainable scale and shift parameters, learned by backprop just like any weight — they exist so the network can, if it needs to, learn to undo the normalization (e.g. recovering the original scale) rather than being forced to keep every activation zero-mean, unit-variance.

5. Residual (Skip) Connections

Lecture 12 named ResNet's 152-layer network the "beginning of the ultra-deep network era" and credited residual connections for making it trainable. The mechanism is simple to state: instead of a block of layers computing a new representation \(F(X)\) from scratch, its input \(X\) is added back to the block's output before the next activation, giving \(F(X)+X\). The input "bypasses" the block via a shortcut path.

6. Building the Network in Keras: A Walkthrough

Putting Sections 1–3 together for a concrete task — MNIST digit classification, 10 classes, 28×28 grayscale input — gives the instructor's own layer choices: two convolution+pool blocks, then flatten, then two dense layers ending in softmax. Applying Lecture 13's output-size formula layer by layer (no padding specified, so Keras defaults to padding='valid', i.e. \(P=0\)) traces exactly how the shape shrinks and the channel depth grows:

Check the first row with Lecture 13's formula: \((28-5)/1+1=24\), giving 24×24, times 32 filters (Section 4 of Lecture 13's "number of filters controls output depth" rule) → 24×24×32. Every other row follows the same formula.

The optimizer is Adam (Lecture 9), and because there are more than two classes the loss is (sparse) categorical cross-entropy (Section 3 above) rather than Lecture 8's binary cross-entropy.

7. Common Pitfalls

8. Summary

9. Code: A Complete Keras CNN for MNIST

This is the instructor's own layer stack from Section 6, written as a complete, correct, runnable Keras script — including data loading, model definition, compilation with Adam (Lecture 9) and sparse categorical cross-entropy (Section 3), and a model.fit call. A full training run is heavy for a CPU; the comments note expected CPU vs. GPU training time so a student can decide how many epochs to actually run.

"""
Lecture 14 -- CNN in Practice: Flattening, FC Layers, Softmax & Implementation
A complete, runnable Keras CNN for MNIST digit classification (10 classes,
28x28x1 input), using the instructor's own layer choices:
Conv2D(32,5x5) -> MaxPool(2x2) -> Conv2D(64,5x5) -> MaxPool(2x2)
-> Flatten -> Dense(1000, relu) -> Dense(num_classes, softmax)

CPU training note: ~1-2 minutes/epoch on a typical laptop CPU for MNIST.
GPU (local or Colab) note: a few seconds/epoch -- prefer GPU for anything
beyond a quick correctness check. See resources.html for a link to the
official TensorFlow CNN tutorial (Colab-enabled) for a full training run.
"""
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dense
from tensorflow.keras.datasets import mnist
from tensorflow.keras.utils import to_categorical  # not used with sparse CE, kept for reference

# ---- data: MNIST handwritten digits ----
(x_train, y_train), (x_test, y_test) = mnist.load_data()
input_shape = (28, 28, 1)
num_classes = 10

x_train = x_train.reshape(-1, 28, 28, 1).astype("float32") / 255.0
x_test  = x_test.reshape(-1, 28, 28, 1).astype("float32") / 255.0
# labels stay as plain integers 0-9 -- matches sparse_categorical_crossentropy below

# ---- model: instructor's layer stack ----
model = Sequential()
model.add(Conv2D(32, kernel_size=(5, 5), strides=(1, 1), activation='relu', input_shape=input_shape))
model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))
model.add(Conv2D(64, (5, 5), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(1000, activation='relu'))
model.add(Dense(num_classes, activation='softmax'))

model.summary()

# ---- compile: Adam optimizer (Lecture 9) + categorical cross-entropy (Section 3) ----
model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

# ---- train ----
# NOTE: run a small number of epochs / a data subset for a quick CPU check;
# see the box in Lecture 14 Section 6 for why a full run belongs on GPU.
model.fit(x_train, y_train,
          batch_size=128,
          epochs=5,
          validation_split=0.1)

test_loss, test_acc = model.evaluate(x_test, y_test, verbose=0)
print(f"Test accuracy: {test_acc:.4f}")

⬇ Download lecture-14-cnn-keras.py   More resources for this lecture →