Hi,
I think it could be helpful to add some figures for the loss in the assignment. This is for the 2nd assignment of the 1st week.
It was unclear to me the purpose of the “smooth” function in utils.py until I plotted the unsmoothed and smoothed losses in the “model” function.
Here is the unsmoothed:
And here goes the smoothed:
What we observe in the first figure is the loss for stochastic gradient descent. And it is normal for it to be oscillatory. But we cannot discern any patterns from it, so we go for the smoothing, which is an exponentially weighted average.
Without plotting, it was unclear to me the idea behind smoothing because the only unsmoothed loss was used in backpropagation. However, the smoothed loss is nevertheless no less important because, without it, we cannot see if the model is being optimized or not.
This is my understanding and suggestion.
I hope it makes sense.
Henrikh