Why does the training accuracy go up and then down? (Lab 1)

Course Q&A TensorFlow Developer Professional Introduction to TF for Artificial Intelligence ...
1

Lab 1 Horses or humans…

As the title says.

According to the notes this is probably due to a too-large step down the gradient, moving back and forth across minima. But this is a demo so I didn’t expect this behaviour?

Epoch 1/15
8/8 [==============================] - 14s 740ms/step - loss: 1.7635 - accuracy: 0.6107
Epoch 2/15
8/8 [==============================] - 6s 717ms/step - loss: 0.6281 - accuracy: 0.5940
Epoch 3/15
8/8 [==============================] - 6s 743ms/step - loss: 0.4198 - accuracy: 0.8454
Epoch 4/15
8/8 [==============================] - 7s 832ms/step - loss: 0.3556 - accuracy: 0.8682
Epoch 5/15
8/8 [==============================] - 6s 732ms/step - loss: 0.4360 - accuracy: 0.8276
Epoch 6/15
8/8 [==============================] - 6s 727ms/step - loss: 0.1798 - accuracy: 0.9299
Epoch 7/15
8/8 [==============================] - 6s 717ms/step - loss: 0.1129 - accuracy: 0.9611
Epoch 8/15
8/8 [==============================] - 6s 729ms/step - loss: 0.6208 - accuracy: 0.8521
Epoch 9/15
8/8 [==============================] - 6s 723ms/step - loss: 0.1002 - accuracy: 0.9566
Epoch 10/15
8/8 [==============================] - 6s 721ms/step - loss: 0.0601 - accuracy: 0.9800
Epoch 11/15

2

Where in the notes does the comment about step size appear?

3

I was inaccurate. Not in this notebook. But it has been discussed in this course briefly, and I’ve read it elsewhere.

I’d just like a comment whether what I am seeing is to be expected, and if so why it should be.

Tx

4

Your obseration of the accuracy bouncing close to 100 % is correct. There are ways to change learning rate dynamically (see ExponentialDecay ) to reduce step size during later epochs. You’ll learn about a couple of important callbacks to help pick the best performing model as you progress in the specialization.

1 Like