Number of Parameters in a CNN model

Course Q&A Deep Learning Specialization Convolutional Neural Networks
,
1

Can someone confirm if the following computation shown in the figure is wrong?

The number of trainable parameters in a layer of the LeNet-5 Architecture:

image
image876×596 59.8 KB

Image Source - Medium Article

This same value is given by someone on Kaggle.

I think the corrected value of the trainable parameters for this specific layer - all weights and biases in all kernels - should be:
(5 * 5 * 6 * 16) + 16 = 2416.

Please let me know which one is correct? 2416 or 1516.

3 Likes
2

The highlighed layer has 5x5 filters and the input has 6 channels. So, the number of trainable parameters for each filter is 5x5x6 + 1 (the 1 is for bias). This makes it 151 parameters per filter. Since we have 16 filters, the total number of trainable parameters is 151*16 = 2416

Here’s lenet in code:

import tensorflow as tf

lenet = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(32,32,1)),
    tf.keras.layers.Conv2D(filters=6, kernel_size=5, strides=1, activation='tanh'),
    tf.keras.layers.AveragePooling2D(strides=2, pool_size=2),
    tf.keras.layers.Activation('tanh'),
    tf.keras.layers.Conv2D(filters=16, kernel_size=5, strides=1, activation='tanh'),
    tf.keras.layers.AveragePooling2D(strides=2, pool_size=2),
    tf.keras.layers.Activation('tanh'),
    tf.keras.layers.Conv2D(filters=120, kernel_size=5, strides=1, activation='tanh'),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(units=84, activation='tanh'),
    tf.keras.layers.Dense(units=10, activation='softmax'),
])
lenet.summary()

Output

Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ conv2d (Conv2D)                 │ (None, 28, 28, 6)      │           156 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ average_pooling2d               │ (None, 14, 14, 6)      │             0 │
│ (AveragePooling2D)              │                        │               │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ activation (Activation)         │ (None, 14, 14, 6)      │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_1 (Conv2D)               │ (None, 10, 10, 16)     │         2,416 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ average_pooling2d_1             │ (None, 5, 5, 16)       │             0 │
│ (AveragePooling2D)              │                        │               │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ activation_1 (Activation)       │ (None, 5, 5, 16)       │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_2 (Conv2D)               │ (None, 1, 1, 120)      │        48,120 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ flatten (Flatten)               │ (None, 120)            │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense (Dense)                   │ (None, 84)             │        10,164 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_1 (Dense)                 │ (None, 10)             │           850 │
└─────────────────────────────────┴────────────────────────┴───────────────┘

 Total params: 61,706 (241.04 KB)
 Trainable params: 61,706 (241.04 KB)
 Non-trainable params: 0 (0.00 B)
5 Likes