Week 3 - Assignment 3.3 (epoch_cost)

Course Q&A Deep Learning Specialization Improving Deep Neural Networks: Hyperparameter tun
1

In the assignment when computing the epoch_cost, why is the following line there?

    # We divide the epoch cost over the number of samples
   epoch_cost /= m

From the code, we are already computing the mean per minibatch (given from our tf.reduce_mean in the cost function).

Why exactly are we even accumulating? If anything, we should just be resetting and printing the new cost after every epoch (over time should be reducing the cost anyways)

2

Hi @xmkhan,

This is an interesting question. The division operation is there to smooth out fluctuations in the minibatch costs, so that we get one final cost per epoch.

At the end of the minibatch loop, epoch_cost is the sum of all mean minibatch costs in an epoch. Therefore, it could have been more exact to divide epoch_cost by the number of minibatches, rather than the number of all training examples (m). The net effect seems to be making epoch_cost smaller than it actually is, by roughly the minibatch size (32 by default).

Nevertheless, the training cost should go down and the training accuracy should go up as the epoch number increases.