Week 2 - Exercise 2 - Mini Batch

Course Q&A Deep Learning Specialization Improving Deep Neural Networks: Hyperparameter tun
1

Hi, I am having some difficulty coding for the last mini-batch. I know what to do. I should start from the end until the mini-batch size condition is fulfilled (e.g., If mini-batch size is 64, I should start from the end of the examples and continue until the size is 64) but I am unable to construct the programming logic for it. Can anyone help please? Thank You!

2

In terms of creating mini-batches the logic should be, you start from the beggining till you reach 64 and then from 65 till you reach 128 and so on and so on, and on the last mini-batch you take what its left till the end of the entire list of examples. You can do it from the end too, shouldnt make much difference.

There is though Tensorflow code that does this automatically.

3

How do I make the logic to start from the beginning of the end of the previous mini-batch? I want to start from there and go until the end as I’ve now realized that we need the remaining (in this case 20) elements in the last mini batch only

4

Hey @Syed_Hamza_Tehseen,
For building this logic, 4 variables will come in handy to you:

  • m = Total number of samples, which you can use for defining the last mini-batch
  • mini_batch_size = Maximum number of samples in a mini-batch, which you can use to find the indices of the various mini-batches.
  • num_complete_minibatches = Denotes the total number of mini-batches, consisting of samples equal to mini_batch_size
  • k = Denotes the indices for the mini-batches

Note that all of these variables have already been defined for you. Now, let’s say that k = 2 and mini_batch_size = 32. Since k = 2, this means we are referring to the 3rd mini-batch (since indexing begins from 0 in Python). Now in this case, the first mini-batch will have samples indexed from 0 to 31. Similarly, the second mini-batch will have samples indexed from 32 to 63.

Now, the 3rd mini-batch, which we are concerned about has samples indexed from 64 to 95. If we contemplate k * mini_batch_size = 64 and (k * mini_batch_size) - 1 = 95. Voila, our indices :partying_face: Cherry on the cake, when you use slicing for indexing in Python, it doesn’t include the element corresponding to the closing index, i.e., if you use X[start, end], this won’t include the sample X[end]. Now, I leave it up to you to figure out the code for the last mini-batch.

Cheers,
Elemento

1 Like