Mini Batch Gradient Descent vs Batch GD

Batch GD means for 1 epoch, you have to parse through the entire training set and then update weights or parameters.
In Mini Batch GD, for 1 epoch, if there are 1,00,000 examples, batch_size=100,so each batch gets 1000 examples each,
The weights are being updates after each batch(after every 1000 examples)

This is my understading, Please correct me If I am wrong!
Thanks and regards

Hi @ajaykumar3456,

Your understanding is correct.
A small note on the Mini Batch GD: If there are 100,000 examples and your batch_size is 100. Weights are being updated after looking into 100 examples and there will be 1000 steps. Though, it’s a good practice to start with lower batch sizes.


1 Like

batch_size=100 means 100 examples per batch. Thanks for your clarification. what i thought was batch_size=100 means 100 batches, each batch 1000 examples.
Thanks for your clarification

1 Like