Mini Batch Gradient Descent vs Batch GD

Course Q&A Deep Learning Specialization Improving Deep Neural Networks: Hyperparameter tun
1

Batch GD means for 1 epoch, you have to parse through the entire training set and then update weights or parameters.
In Mini Batch GD, for 1 epoch, if there are 1,00,000 examples, batch_size=100,so each batch gets 1000 examples each,
The weights are being updates after each batch(after every 1000 examples)

This is my understading, Please correct me If I am wrong!
Thanks and regards

2

Hi @ajaykumar3456,

Your understanding is correct.
A small note on the Mini Batch GD: If there are 100,000 examples and your batch_size is 100. Weights are being updated after looking into 100 examples and there will be 1000 steps. Though, it’s a good practice to start with lower batch sizes.

Best,
Bahadir

1 Like
3

batch_size=100 means 100 examples per batch. Thanks for your clarification. what i thought was batch_size=100 means 100 batches, each batch 1000 examples.
Thanks for your clarification
@bahadir

1 Like