Determinig Batch Size

Course Q&A TensorFlow Developer Professional Convolutional Neural Networks in TensorFlow
1

I am having a doubt while deciding the batch size for both the training and validation datasets!
How should I determine if it should be 20/32/50/128,etc?

2

Try the different values and see which one works best.

Then there’s the famous quote from Yann LeCun: “Friends don’t let friends use batch sizes greater than 32.” :nerd_face: Here’s the link to the Twitter post with the full quote.

1 Like
3

Haha… witty of you!
Well…Thank you so much!

4

Note that the Twitter thread included a link to a paper by LeCun’s group showing the results of a lot of experiments and concluding that over a pretty wide range of deep learning training tasks, batch sizes between m = 2 and m = 32 consistently gave the best performance.

So at least that limits your search space. :laughing:

1 Like