All of my lines/codes got “all tests pass”, but I don’t know why my model’s accuracy starts decreasing after around epoch 4 while the expected output keeps increasing? Thank you so much!
It turns out there is never any guarantee of convergence with these models. Note that they do not set the TF seeds here, so you can also get different behavior from run to run. The solution surfaces are very complex and if you start at a different random point in the multidimensional space, your path will be different.
1 Like
Notice that even the expected accuracy doesn’t actually ‘keep increasing’, at least not monotonically. More sophisticated approaches to training might include making checkpoints along the way and evaluating accuracy trend at each epoch to possibly trigger an early exit if it seems like the optimization has lost its way.
1 Like