For the assignment C2W2A1 , if we increase the epochs we get these peaks at intervals in cost v/s iteration plot with continuing descend afterwards. What is the reason behind this and is there any way to optimize this ?
Sorry, for some reason I no longer have access to that lab, so I can’t run it to investigate this issue.
Hi,
If I restart the kernel after each change made to the number of epochs, everything goes smooth. Without restarting the kernel, the result seems erroneous indeed.
Steps:
- kernel restart after the first run of
plot_loss_tf(history) - epochs = 200
- run all
- result OK
- no kernel restart
- epochs = 100
- rerun
plot_loss_tf(history) - result NOK
2 Likes
In general, the notebooks contain a lot of global variables, and if you just change some aspect of the model configuration, and then train it again, that won’t always reset the workspace fully.
FYI, that’s a big cause of weird behavior in some of the other courses (like the Deep Learning Specialization).
But that is a strange bit of behavior that might be worth exploring further. Perhaps it has to do with exactly what plot_loss_tf() is doing.
2 Likes
Indeed, if we rerun the cell containing model = Sequential everything goes OK, too. So the class keras.models.Sequential needs to be re-instantiated. Note: without re-instantiating it, the loss keeps going down from where it stopped previously, kind of epochs+= new_num_epochs
1 Like
It’s because the weight values are initialized only when the model object is created.
3 Likes