Hi all,
I have finished week2 of DLS2 and I am confused over the definition of epoch and iteration.
Does epoch only apply to mini-batch?
Epoch is a pass through the data. Isn’t iteration the same?
Can anyone clarify please? It might be easier to visualize with a drawing.
Thanks a lot!
Iteration is a more generic term: it just means one pass through a loop, whatever that loop does. Epoch is specific to minibatch gradient descent: as you say, it means one pass through the entire training dataset (all minibatches). If you are doing full batch GD, there is one loop and one iteration of that loop covers all the data (one Epoch). But in minibatch, there are two loops, right? The outer loop is over the Epochs and the inner loop processes one minibatch per iteration. So the definition of “iteration” becomes a bit ambiguous in that case, but I think normally people would assume it refers to the outer loop. Meaning that people are a bit sloppy sometimes and use “epoch” and “iteration” interchangeably, but it’s better to stick with epoch for clarity.
Thank you for clearing this up @paulinpaloalto