Chapter 13: Convolutional Neural Networks (CNNs)

Bloom's Taxonomy Map for This Chapter

Learning Objectives

By the end of this chapter, you will be able to:

• Explain why flattening a 224×224×3 image creates 150,528 inputs per neuron, and how local connectivity + weight sharing in CNNs reduces this by 2000×
• Derive the 2D cross-correlation (convolution) formula from scratch, implement it in NumPy, and compute output feature map dimensions: ⌊(W − F + 2P) / S⌋ + 1
• Distinguish between Valid, Same, and Full padding modes, and between Max pooling and Average pooling with their respective use cases
• Trace the evolution of CNN architectures from LeNet-5 (60K params) → AlexNet (61M) → VGGNet (138M) → GoogLeNet (6.8M) → ResNet (25.6M) → EfficientNet (5.3M), explaining the key innovation in each
• Prove mathematically why ResNet skip connections solve the degradation problem by ensuring gradient flow: ∂L/∂x = ∂L/∂y · (1 + ∂F/∂x)
• Build a complete CNN from scratch using NumPy (forward + backward pass), then rebuild it in PyTorch for MNIST classification
• Apply Grad-CAM visualization to interpret what a CNN has learned, connecting activation maps to human-interpretable features
• Design a transfer learning pipeline using a pretrained ResNet-50 for Indian soil type classification

Opening Hook

The Intuition First

The "Flashlight in a Dark Room" Analogy

Imagine you walk into a completely dark room and you need to understand what's inside. You have two options:

• Option A (Fully Connected): Turn on every light in the room at once. You see everything simultaneously — but you're overwhelmed. Every pixel of your visual field connects to every neuron in your brain. For a 224×224 room, that's 150,528 connections per neuron. Your brain would melt.
• Option B (Convolutional): Use a small flashlight and scan the room systematically. At each position, you examine a small 3×3 patch. You look for the same features everywhere — edges, corners, textures. You use the same flashlight (same weights) at every position. This is a CNN.

The flashlight analogy captures the two key ideas of CNNs:

1. Local connectivity: Each neuron only "sees" a small patch of the input (its receptive field), not the entire image
2. Weight sharing: The same filter (flashlight) is used at every spatial position — an edge detector that works in the top-left also works in the bottom-right

The "Aha!" Question

🤔 If a 3×3 filter can only see 9 pixels, how can a CNN recognize an entire cat that spans hundreds of pixels? The answer is the receptive field hierarchy — the same reason you can understand this entire sentence even though your eye fixation only reads ~7 characters at a time.

Layer 1 sees 3×3 patches. Layer 2 sees each of those 3×3 patches, effectively covering 5×5 of the original image. Layer 3 covers 7×7. By stacking layers, small local views compose into global understanding. A 50-layer network with 3×3 filters has a theoretical receptive field larger than any practical image. This is the deep magic of CNNs: local operations, global understanding.

Why CNNs? The Parameter Explosion Problem

The Fully Connected Catastrophe

Let's do some arithmetic that will make you feel the problem. Consider a standard color image: 224 × 224 × 3 = 150,528 pixels. If you flatten this into a vector and connect it to a fully connected hidden layer of 1,000 neurons:

For context, the entire AlexNet (which won ImageNet in 2012) has 61 million parameters total. A single fully connected layer on a 224×224 image would need 2.5× more parameters than AlexNet.

This creates three devastating problems:

1. Memory: 150M float32 parameters = 600MB of memory — for one layer!
2. Overfitting: With 150M parameters, you'd need hundreds of millions of training images to avoid memorizing the data
3. Wasted Structure: A fully connected layer treats the pixel at position (0,0) as equally related to the pixel at (223,223). But in images, nearby pixels are related, distant pixels usually aren't. The FC layer ignores this spatial structure entirely.

CNN's Two Key Insights

The Convolution Operation (Derived from Scratch)

Mathematical Definition

What deep learning calls "convolution" is technically cross-correlation. True mathematical convolution flips the kernel; in practice, since the kernel weights are learned, flipping doesn't matter — the network learns the flipped version if needed.

1D Cross-Correlation (Warmup)

Given input signal x of length W and filter f of length F, the output y at position i is:

2D Cross-Correlation (The Real Thing)

For a 2D input X of size H × W and filter K of size F_h × F_w:

where P = padding, S = stride, and b is the bias term.

Worked Example: 2D Convolution by Hand

Let's convolve a 4×4 input with a 3×3 filter (stride=1, padding=0):

Multi-Channel Convolution (3D)

Real images have 3 channels (RGB). The filter must also have 3 channels. The convolution is done channel-wise and summed:

Key insight: one filter produces one 2D feature map. To detect multiple features (edges, textures, colors), you use multiple filters. If you use K filters, the output has K channels.

NumPy Implementation of 2D Convolution

Python / NumPy
import numpy as np

def conv2d(X, K, stride=1, padding=0):
    """
    2D cross-correlation (what DL calls 'convolution').
    X: input array of shape (H, W)
    K: kernel of shape (Fh, Fw)
    Returns: output array of shape (H_out, W_out)
    """
    H, W = X.shape
    Fh, Fw = K.shape

    # Step 1: Pad the input
    if padding > 0:
        X = np.pad(X, padding, mode='constant', constant_values=0)
        H, W = X.shape  # Update dimensions after padding

    # Step 2: Compute output dimensions
    H_out = (H - Fh) // stride + 1
    W_out = (W - Fw) // stride + 1

    # Step 3: Initialize output
    Y = np.zeros((H_out, W_out))

    # Step 4: Slide the filter and compute dot products
    for i in range(H_out):
        for j in range(W_out):
            # Extract the receptive field
            patch = X[i*stride : i*stride + Fh,
                      j*stride : j*stride + Fw]
            # Element-wise multiply and sum (dot product)
            Y[i, j] = np.sum(patch * K)

    return Y

# Test with our worked example
X = np.array([[1,2,3,0],
              [0,1,2,3],
              [3,0,1,2],
              [2,1,0,1]])

K = np.array([[ 1, 0,-1],
              [ 1, 0,-1],
              [ 1, 0,-1]])

result = conv2d(X, K, stride=1, padding=0)
print(result)

Multi-Channel Convolution in NumPy

Python / NumPy
def conv2d_multichannel(X, K, b, stride=1, padding=0):
    """
    Multi-channel 2D convolution.
    X: (C_in, H, W)   — input with C_in channels
    K: (C_out, C_in, Fh, Fw) — filters
    b: (C_out,)        — biases
    Returns: (C_out, H_out, W_out)
    """
    C_out, C_in, Fh, Fw = K.shape
    _, H, W = X.shape

    # Pad spatial dimensions only
    if padding > 0:
        X = np.pad(X, ((0,0), (padding,padding), (padding,padding)),
                   mode='constant')
        _, H, W = X.shape

    H_out = (H - Fh) // stride + 1
    W_out = (W - Fw) // stride + 1
    Y = np.zeros((C_out, H_out, W_out))

    for f in range(C_out):          # For each filter
        for i in range(H_out):
            for j in range(W_out):
                patch = X[:, i*stride:i*stride+Fh,
                             j*stride:j*stride+Fw]
                Y[f, i, j] = np.sum(patch * K[f]) + b[f]
    return Y

# Example: RGB input 4×4, two 3×3 filters
X_rgb = np.random.randn(3, 4, 4)   # 3 channels, 4×4
K_rgb = np.random.randn(2, 3, 3, 3) # 2 filters, 3 ch, 3×3
b_rgb = np.zeros(2)
out = conv2d_multichannel(X_rgb, K_rgb, b_rgb)
print(f"Output shape: {out.shape}")  # (2, 2, 2)

Padding and Stride

Why Padding?

Without padding, two problems emerge:

1. Shrinking output: A 32×32 input with a 5×5 filter gives 28×28 output. After 5 layers: 12×12. Your spatial information disappears.
2. Border pixels ignored: Corner pixels participate in only 1 convolution; center pixels participate in F×F convolutions. The borders are underrepresented.

Three Padding Modes

Stride: Controlling the Step Size

Stride controls how many pixels the filter moves at each step. Stride=1 means the filter slides one pixel at a time (maximum overlap). Stride=2 means it jumps 2 pixels, halving the output size. Stride acts as a built-in downsampling operation.

Pooling Layers

What Pooling Does

Pooling reduces the spatial dimensions of the feature map, providing three benefits:

1. Dimensionality reduction: 2×2 max pooling with stride 2 halves each spatial dimension, reducing computation by 4×
2. Translation invariance: Small shifts in the input produce the same pooled output. If a cat's eye moves 1 pixel left, max pooling still selects the same maximum activation.
3. Larger receptive field: By reducing spatial dimensions, subsequent convolutions effectively "see" a larger portion of the original image

Max Pooling vs Average Pooling

Python / NumPy
def max_pool2d(X, pool_size=2, stride=2):
    """Max pooling on a 2D input."""
    H, W = X.shape
    H_out = (H - pool_size) // stride + 1
    W_out = (W - pool_size) // stride + 1
    Y = np.zeros((H_out, W_out))

    for i in range(H_out):
        for j in range(W_out):
            patch = X[i*stride : i*stride + pool_size,
                      j*stride : j*stride + pool_size]
            Y[i, j] = np.max(patch)
    return Y

def avg_pool2d(X, pool_size=2, stride=2):
    """Average pooling on a 2D input."""
    H, W = X.shape
    H_out = (H - pool_size) // stride + 1
    W_out = (W - pool_size) // stride + 1
    Y = np.zeros((H_out, W_out))

    for i in range(H_out):
        for j in range(W_out):
            patch = X[i*stride : i*stride + pool_size,
                      j*stride : j*stride + pool_size]
            Y[i, j] = np.mean(patch)
    return Y

# Test
X = np.array([[1,3,2,4],
              [2,1,1,2],
              [4,2,3,1],
              [1,3,2,1]], dtype=np.float64)
print("Max Pool:", max_pool2d(X))
print("Avg Pool:", avg_pool2d(X))

Full CNN Architecture

A typical CNN follows this pattern:

Classic CNN Architectures: The Evolution

The history of CNNs is a masterclass in engineering creativity. Each architecture solved a specific problem that the previous one couldn't.

Architecture Timeline

LeNet-5 (1998) — The Grandfather

AlexNet (2012) — The Game Changer

AlexNet's key innovations over LeNet:

1. ReLU activation instead of tanh — 6× faster training
2. Dropout (p=0.5) in FC layers — regularization breakthrough
3. Data augmentation — random crops, horizontal flips, color jittering
4. GPU training — split across 2 GTX 580 GPUs (model parallelism!)
5. Local Response Normalization (LRN) — later replaced by Batch Norm

VGGNet (2014) — The "Deeper with Simplicity" Philosophy

GoogLeNet/Inception (2014) — The Multi-Scale Thinker

What Does a 1×1 Convolution Do?

This is one of the most asked interview questions. A 1×1 convolution:

1. Cross-channel interaction: It mixes information across channels at each spatial position (like a per-pixel fully connected layer)
2. Dimensionality reduction: Reducing 256 channels to 64 with a 1×1 conv saves massive computation before expensive 3×3 or 5×5 convolutions
3. Adding non-linearity: With a ReLU after it, a 1×1 conv adds an extra non-linear transformation

ResNets & Skip Connections: The Depth Revolution

The Degradation Problem

Before ResNets, a strange phenomenon puzzled researchers: deeper networks performed worse than shallower ones, even on training data. This was not overfitting (training accuracy was also worse). A 56-layer network had higher training error than a 20-layer network.

This is paradoxical. In theory, a 56-layer network should do at least as well as a 20-layer one — the extra 36 layers could just learn identity mappings (pass the input through unchanged). But standard networks struggle to learn identity mappings through stacks of nonlinear layers.

The ResNet Solution: Skip Connections

He et al. (2015) proposed an elegantly simple fix: instead of learning the desired mapping H(x) directly, learn the residual F(x) = H(x) − x, and add the input back:

Why Skip Connections Work: Gradient Flow Proof

Residual Block Variants

Grad-CAM: Seeing What the CNN Sees

The Interpretability Problem

A CNN gives you a prediction: "This is a cat with 97% confidence." But why does it think it's a cat? Is it looking at the cat's face? Its ears? Or the sofa behind it? Grad-CAM (Gradient-weighted Class Activation Mapping) answers this question.

How Grad-CAM Works

1. Forward pass: Run the image through the network, get the score for the target class y^c
2. Backward pass: Compute gradients of y^c with respect to the feature maps A^k of the last convolutional layer
3. Global Average Pool the gradients: α_k^c = (1/Z) Σ_i Σ_j ∂y^c/∂A^k_ij
4. Weighted combination: L_Grad-CAM^c = ReLU(Σ_k α_k^c · A^k)
5. Upsample to input image size and overlay as a heatmap

Python / PyTorch
import torch
import torch.nn.functional as F
from torchvision import models, transforms
from PIL import Image
import numpy as np

def grad_cam(model, img_tensor, target_class, target_layer):
    """Compute Grad-CAM heatmap for a given class and layer."""
    activations = {}
    gradients = {}

    # Register hooks to capture activations and gradients
    def forward_hook(module, inp, out):
        activations['value'] = out.detach()

    def backward_hook(module, grad_in, grad_out):
        gradients['value'] = grad_out[0].detach()

    handle_fwd = target_layer.register_forward_hook(forward_hook)
    handle_bwd = target_layer.register_full_backward_hook(backward_hook)

    # Forward pass
    output = model(img_tensor)
    model.zero_grad()

    # Backward pass for target class
    one_hot = torch.zeros_like(output)
    one_hot[0, target_class] = 1
    output.backward(gradient=one_hot)

    # Compute Grad-CAM
    acts = activations['value']            # (1, K, H, W)
    grads = gradients['value']              # (1, K, H, W)
    weights = grads.mean(dim=(2, 3), keepdim=True)  # (1, K, 1, 1)

    cam = (weights * acts).sum(dim=1, keepdim=True)   # (1, 1, H, W)
    cam = F.relu(cam)                                    # Only positive influence
    cam = F.interpolate(cam, size=img_tensor.shape[2:],
                        mode='bilinear', align_corners=False)
    cam = cam.squeeze().numpy()
    cam = (cam - cam.min()) / (cam.max() - cam.min() + 1e-8)

    handle_fwd.remove()
    handle_bwd.remove()
    return cam

# Usage example
model = models.resnet50(pretrained=True)
model.eval()
target_layer = model.layer4[-1]  # Last conv block of ResNet-50

# cam = grad_cam(model, img_tensor, target_class=281, target_layer=target_layer)
# Overlay 'cam' on the original image using matplotlib with alpha=0.5

Worked Examples

Example 1: By-Hand Dimension & Parameter Calculation

Example 2: 🇮🇳 Aadhaar Face Verification Pipeline

Example 3: 🇺🇸 Tesla Autopilot — Multi-Camera CNN

Complete CNN from Scratch (NumPy)

Let's build a complete CNN — forward AND backward pass — using only NumPy. No PyTorch, no TensorFlow. Just you, Python, and matrix operations.

Python / NumPy — Complete CNN
import numpy as np

# ============================================================
#   LAYER CLASSES — Each implements forward() and backward()
# ============================================================

class Conv2D:
    """2D Convolution Layer with forward and backward pass."""
    def __init__(self, in_channels, out_channels, kernel_size,
                 stride=1, padding=0):
        self.in_c = in_channels
        self.out_c = out_channels
        self.k = kernel_size
        self.stride = stride
        self.padding = padding

        # He initialization
        scale = np.sqrt(2.0 / (in_channels * kernel_size * kernel_size))
        self.W = np.random.randn(out_channels, in_channels,
                                 kernel_size, kernel_size) * scale
        self.b = np.zeros((out_channels, 1))

        # Gradients
        self.dW = np.zeros_like(self.W)
        self.db = np.zeros_like(self.b)

    def forward(self, X):
        """X: (batch, C_in, H, W) → output: (batch, C_out, H_out, W_out)"""
        self.X = X
        N, C, H, W = X.shape
        p = self.padding

        if p > 0:
            self.X_padded = np.pad(X, ((0,0),(0,0),(p,p),(p,p)),
                                   mode='constant')
        else:
            self.X_padded = X

        H_out = (H + 2*p - self.k) // self.stride + 1
        W_out = (W + 2*p - self.k) // self.stride + 1
        out = np.zeros((N, self.out_c, H_out, W_out))

        for i in range(H_out):
            for j in range(W_out):
                h_s = i * self.stride
                w_s = j * self.stride
                patch = self.X_padded[:, :, h_s:h_s+self.k,
                                          w_s:w_s+self.k]
                # patch: (N, C_in, k, k)
                # W: (C_out, C_in, k, k)
                for f in range(self.out_c):
                    out[:, f, i, j] = np.sum(
                        patch * self.W[f], axis=(1,2,3)
                    ) + self.b[f]
        return out

    def backward(self, dout):
        """dout: (batch, C_out, H_out, W_out) → dX: (batch, C_in, H, W)"""
        N, _, H_out, W_out = dout.shape
        dX_padded = np.zeros_like(self.X_padded)
        self.dW = np.zeros_like(self.W)
        self.db = np.zeros_like(self.b)

        for i in range(H_out):
            for j in range(W_out):
                h_s = i * self.stride
                w_s = j * self.stride
                patch = self.X_padded[:, :, h_s:h_s+self.k,
                                          w_s:w_s+self.k]
                for f in range(self.out_c):
                    # dW: accumulate gradient
                    self.dW[f] += np.sum(
                        patch * dout[:, f, i, j].reshape(-1,1,1,1),
                        axis=0)
                    self.db[f] += np.sum(dout[:, f, i, j])
                    # dX: propagate gradient back
                    dX_padded[:, :, h_s:h_s+self.k,
                              w_s:w_s+self.k] += (
                        self.W[f] * dout[:, f, i, j].reshape(-1,1,1,1))

        if self.padding > 0:
            p = self.padding
            return dX_padded[:, :, p:-p, p:-p]
        return dX_padded


class MaxPool2D:
    def __init__(self, pool_size=2, stride=2):
        self.pool = pool_size
        self.stride = stride

    def forward(self, X):
        self.X = X
        N, C, H, W = X.shape
        H_out = (H - self.pool) // self.stride + 1
        W_out = (W - self.pool) // self.stride + 1
        out = np.zeros((N, C, H_out, W_out))
        self.mask = np.zeros_like(X)

        for i in range(H_out):
            for j in range(W_out):
                h_s, w_s = i*self.stride, j*self.stride
                patch = X[:, :, h_s:h_s+self.pool, w_s:w_s+self.pool]
                out[:, :, i, j] = np.max(patch, axis=(2,3))
                # Store mask for backward pass
                max_vals = out[:, :, i, j][:, :, None, None]
                self.mask[:, :, h_s:h_s+self.pool,
                          w_s:w_s+self.pool] += (patch == max_vals)
        return out

    def backward(self, dout):
        N, C, H_out, W_out = dout.shape
        dX = np.zeros_like(self.X)
        for i in range(H_out):
            for j in range(W_out):
                h_s, w_s = i*self.stride, j*self.stride
                dX[:, :, h_s:h_s+self.pool,
                   w_s:w_s+self.pool] += (
                    self.mask[:, :, h_s:h_s+self.pool,
                              w_s:w_s+self.pool]
                    * dout[:, :, i, j][:, :, None, None])
        return dX


class ReLU:
    def forward(self, X):
        self.mask = (X > 0)
        return X * self.mask

    def backward(self, dout):
        return dout * self.mask


class Flatten:
    def forward(self, X):
        self.shape = X.shape
        return X.reshape(X.shape[0], -1)

    def backward(self, dout):
        return dout.reshape(self.shape)


class Dense:
    def __init__(self, in_features, out_features):
        self.W = np.random.randn(in_features, out_features) * np.sqrt(
            2.0 / in_features)
        self.b = np.zeros((1, out_features))
        self.dW = np.zeros_like(self.W)
        self.db = np.zeros_like(self.b)

    def forward(self, X):
        self.X = X
        return X @ self.W + self.b

    def backward(self, dout):
        self.dW = self.X.T @ dout
        self.db = np.sum(dout, axis=0, keepdims=True)
        return dout @ self.W.T


def softmax_cross_entropy(logits, labels):
    """Numerically stable softmax + cross-entropy."""
    shifted = logits - np.max(logits, axis=1, keepdims=True)
    exp_scores = np.exp(shifted)
    probs = exp_scores / np.sum(exp_scores, axis=1, keepdims=True)
    N = logits.shape[0]
    loss = -np.sum(np.log(probs[range(N), labels] + 1e-8)) / N
    dlogits = probs.copy()
    dlogits[range(N), labels] -= 1
    dlogits /= N
    return loss, dlogits


# ============================================================
#   BUILD THE CNN: Conv→ReLU→Pool → Conv→ReLU→Pool → FC→Out
# ============================================================

# Architecture for MNIST (28×28×1 → 10 classes)
layers = [
    Conv2D(in_channels=1, out_channels=8, kernel_size=3, padding=1),
    ReLU(),
    MaxPool2D(pool_size=2, stride=2),      # 28→14
    Conv2D(in_channels=8, out_channels=16, kernel_size=3, padding=1),
    ReLU(),
    MaxPool2D(pool_size=2, stride=2),      # 14→7
    Flatten(),                               # 7×7×16 = 784
    Dense(in_features=784, out_features=10),
]

def forward_pass(X, layers):
    for layer in layers:
        X = layer.forward(X)
    return X

def backward_pass(dout, layers):
    for layer in reversed(layers):
        dout = layer.backward(dout)

def update_params(layers, lr=0.01):
    for layer in layers:
        if hasattr(layer, 'W'):
            layer.W -= lr * layer.dW
            layer.b -= lr * layer.db

# ============================================================
#   TRAINING LOOP (on a small batch for demonstration)
# ============================================================

# Generate synthetic "MNIST-like" data for testing
np.random.seed(42)
X_train = np.random.randn(64, 1, 28, 28) * 0.1
y_train = np.random.randint(0, 10, size=64)

print("Training CNN from scratch with NumPy...")
for epoch in range(5):
    logits = forward_pass(X_train, layers)
    loss, dlogits = softmax_cross_entropy(logits, y_train)
    backward_pass(dlogits, layers)
    update_params(layers, lr=0.01)
    preds = np.argmax(logits, axis=1)
    acc = np.mean(preds == y_train)
    print(f"  Epoch {epoch+1}: loss={loss:.4f}, acc={acc:.2%}")

PyTorch Implementation — MNIST CNN

Python / PyTorch
import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import datasets, transforms
from torch.utils.data import DataLoader

# ─── Step 1: Define the CNN architecture ───
class MNISTConvNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.features = nn.Sequential(
            # Block 1: 28×28×1 → 14×14×32
            nn.Conv2d(1, 32, kernel_size=3, padding=1),
            nn.BatchNorm2d(32),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(2, 2),

            # Block 2: 14×14×32 → 7×7×64
            nn.Conv2d(32, 64, kernel_size=3, padding=1),
            nn.BatchNorm2d(64),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(2, 2),

            # Block 3: 7×7×64 → 3×3×128
            nn.Conv2d(64, 128, kernel_size=3, padding=0),
            nn.BatchNorm2d(128),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(2, 2),            # → 2×2×128 (floor of 5/2)
        )
        self.classifier = nn.Sequential(
            nn.Flatten(),                   # 2×2×128 = 512
            nn.Linear(512, 128),
            nn.ReLU(inplace=True),
            nn.Dropout(0.3),
            nn.Linear(128, 10),
        )

    def forward(self, x):
        x = self.features(x)
        x = self.classifier(x)
        return x

# ─── Step 2: Data loading ───
transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.1307,), (0.3081,))  # MNIST mean/std
])

train_data = datasets.MNIST('./data', train=True,
                            download=True, transform=transform)
test_data = datasets.MNIST('./data', train=False, transform=transform)
train_loader = DataLoader(train_data, batch_size=128, shuffle=True)
test_loader = DataLoader(test_data, batch_size=256)

# ─── Step 3: Training ───
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = MNISTConvNet().to(device)
optimizer = optim.Adam(model.parameters(), lr=1e-3)
criterion = nn.CrossEntropyLoss()

print(f"Model parameters: {sum(p.numel() for p in model.parameters()):,}")

for epoch in range(5):
    model.train()
    total_loss = 0
    for batch_x, batch_y in train_loader:
        batch_x, batch_y = batch_x.to(device), batch_y.to(device)
        optimizer.zero_grad()
        output = model(batch_x)
        loss = criterion(output, batch_y)
        loss.backward()
        optimizer.step()
        total_loss += loss.item()

    # Evaluate
    model.eval()
    correct = 0
    with torch.no_grad():
        for batch_x, batch_y in test_loader:
            batch_x, batch_y = batch_x.to(device), batch_y.to(device)
            preds = model(batch_x).argmax(dim=1)
            correct += (preds == batch_y).sum().item()
    acc = correct / len(test_data)
    print(f"Epoch {epoch+1}: loss={total_loss/len(train_loader):.4f}, "
          f"test_acc={acc:.2%}")

class BrokenCNN(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(1, 16, 5)      # 28→24
        self.pool = nn.MaxPool2d(2, 2)         # 24→12
        self.conv2 = nn.Conv2d(16, 32, 5)     # 12→8
        # pool: 8→4
        self.fc1 = nn.Linear(32 * 5 * 5, 10)  # ← BUG HERE!

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = x.view(x.size(0), -1)
        return self.fc1(x)

Visual Diagrams

Feature Hierarchy Visualization

ResNet-50 Architecture

Case Study: Aadhaar Face Authentication 🇮🇳

Case Study: Tesla Autopilot 🇺🇸

Common Misconceptions

GATE/Exam Corner

Formula Sheet

GATE Previous Year Questions (Pattern)

Prediction Table: What GATE Will Ask Next

Interview Prep

Conceptual Questions

Coding Questions

System Design Case Study

Hands-On Lab: Indian Soil Type Classification

Python / PyTorch
import torch
import torch.nn as nn
from torchvision import models, transforms
from torchvision.datasets import ImageFolder
from torch.utils.data import DataLoader

# ─── Data Augmentation (crucial for small datasets!) ───
train_transform = transforms.Compose([
    transforms.RandomResizedCrop(224, scale=(0.7, 1.0)),
    transforms.RandomHorizontalFlip(),
    transforms.RandomVerticalFlip(),       # Soil can be any orientation
    transforms.ColorJitter(brightness=0.3, contrast=0.3,
                           saturation=0.3, hue=0.1),
    transforms.RandomRotation(30),
    transforms.ToTensor(),
    transforms.Normalize([0.485,0.456,0.406], [0.229,0.224,0.225])
])

val_transform = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize([0.485,0.456,0.406], [0.229,0.224,0.225])
])

# ─── Transfer Learning: Freeze backbone, replace head ───
model = models.resnet50(weights=models.ResNet50_Weights.DEFAULT)

# Freeze all backbone layers
for param in model.parameters():
    param.requires_grad = False

# Replace final FC layer (1000 → 6 soil types)
num_features = model.fc.in_features   # 2048
model.fc = nn.Sequential(
    nn.Linear(num_features, 256),
    nn.ReLU(),
    nn.Dropout(0.4),
    nn.Linear(256, 6)               # 6 soil types
)

# Only FC layers have requires_grad=True → much faster training!
trainable = sum(p.numel() for p in model.parameters() if p.requires_grad)
print(f"Trainable params: {trainable:,}")  # ~525K out of 25.6M

# ─── Fine-tuning Strategy ───
# Phase 1: Train only FC (5 epochs, lr=1e-3)
# Phase 2: Unfreeze last 2 ResNet blocks + FC (10 epochs, lr=1e-4)
# Phase 3: Unfreeze all (5 epochs, lr=1e-5, with cosine annealing)

optimizer = torch.optim.Adam(
    filter(lambda p: p.requires_grad, model.parameters()),
    lr=1e-3
)
scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=20)
criterion = nn.CrossEntropyLoss()

Exercises

Section A: Conceptual Questions (5)

Section B: Mathematical Problems (8)

Section C: Coding Exercises (4)

Section D: Critical Thinking (3)

★ Starred Research Questions (2)

Connections

Chapter Summary

Further Reading

🇮🇳 Indian Resources

• NPTEL: "Deep Learning" by Prof. Mitesh Khapra (IIT Madras) — Weeks 7-8 cover CNNs with Indian examples
• NPTEL: "Computer Vision" by Prof. Vineeth N. Balasubramanian (IIT Hyderabad) — CNN architectures in detail
• GATE CS: Previous year questions on CNN output size computation and parameter counting (2019-2024)
• Book: "Deep Learning" by Goodfellow, Bengio, Courville — Chapter 9: Convolutional Networks (standard GATE reference)
• IITD CVIT Lab: Research papers on Indian face recognition and document analysis using CNNs

🌐 Global Resources

• Original Papers (Must-Read):
  • LeCun et al. (1998) — "Gradient-Based Learning Applied to Document Recognition" (LeNet)
  • Krizhevsky et al. (2012) — "ImageNet Classification with Deep CNNs" (AlexNet)
  • He et al. (2015) — "Deep Residual Learning for Image Recognition" (ResNet)
  • Tan & Le (2019) — "EfficientNet: Rethinking Model Scaling for CNNs"
  • Liu et al. (2022) — "A ConvNet for the 2020s" (ConvNeXt)
• 3Blue1Brown: "But what is a convolution?" — Beautiful visual explanation of the convolution operation
• Distill.pub: "Feature Visualization" (Olah et al.) — Interactive exploration of what CNN layers learn
• CS231n (Stanford): Lecture 5 (Convolutional Neural Networks) — Fei-Fei Li's gold-standard course
• Grad-CAM Paper: Selvaraju et al. (2017) — "Grad-CAM: Visual Explanations from Deep Networks"
• Blog: "An Intuitive Explanation of Convolutional Neural Networks" — ujjwalkarn.me

🔬 Cutting-Edge (2023-2025)

• InternImage (2023) — Deformable convolutions at scale, competing with ViT
• RepLKNet (2022) — Very large kernels (31×31) in CNNs, revisiting the VGG small-kernel assumption
• FlexiViT (2023) — Flexible patch size Vision Transformers, bridging CNN and ViT paradigms
• EfficientNetV2 (2021) — Progressive resizing + Fused-MBConv for faster training