Week 2: Optimization Methods

Course Q&A Deep Learning Specialization Improving Deep Neural Networks: Hyperparameter tun
1

  1. 2.1
    2.1715×24 6.86 KB

  2. 2
    2716×43 11.5 KB

I have been trying to figure out this slicing done here.
Can you explain fig. 1 and 2 respectively?

Note: mini_batch_size (here 64)
Thank You.

2 Likes
2

Hi @starboy,

Maybe would be helpful if you compute those expressions in a python interpreter and see what values are you getting to better understand the slicing. However, I’m gonna try to explain it in simple terms.

  1. The : in the first part says give me all elements from this dimension. The k in the second part is the mini_batch number, so for the first batch k=0, and the slice goes from 0 (0*64) to 64 (1*64), when k=1 tha batch goes from 64 (1*64) to 128 (2*64) and so on. In that way, for each iteration you have a batch of the corresponding batch size, until you arrive at the last batch which doesn’t necessarily have sufficient elements for a complete batch, which lead us to:

  2. Assuming the previous iteration goes num_complete_minibatches times, then the slicing num_complete_minibatches * mini_batch_size: goes from the first element after the processed mini batches to the end of the array (the residual elements that doesn’t make a complete mini batch).

Please let me know if I understood your question correctly and if the response addresses it.

4 Likes
3

@kampamocha thank you so much for such a beautiful explanation.

1 Like