Trouble with overfitting

Course Q&A TensorFlow Developer Professional Natural Language Processing in TensorFlow
1

[code removed - moderator]

Epoch 1/20
4500/4500 [==============================] - 13s 3ms/step - loss: 0.5783 - accuracy: 0.6943 - val_loss: 0.5418 - val_accuracy: 0.7297
Epoch 2/20
4500/4500 [==============================] - 12s 3ms/step - loss: 0.5357 - accuracy: 0.7312 - val_loss: 0.5232 - val_accuracy: 0.7425
Epoch 3/20
4500/4500 [==============================] - 12s 3ms/step - loss: 0.5191 - accuracy: 0.7437 - val_loss: 0.5282 - val_accuracy: 0.7334
Epoch 4/20
4500/4500 [==============================] - 12s 3ms/step - loss: 0.5098 - accuracy: 0.7520 - val_loss: 0.5381 - val_accuracy: 0.7446
Epoch 5/20
4500/4500 [==============================] - 12s 3ms/step - loss: 0.5041 - accuracy: 0.7576 - val_loss: 0.5282 - val_accuracy: 0.7413
Epoch 6/20
4500/4500 [==============================] - 12s 3ms/step - loss: 0.4987 - accuracy: 0.7605 - val_loss: 0.5466 - val_accuracy: 0.7396
Epoch 7/20
4500/4500 [==============================] - 12s 3ms/step - loss: 0.4951 - accuracy: 0.7646 - val_loss: 0.5412 - val_accuracy: 0.7381
Epoch 8/20
4500/4500 [==============================] - 12s 3ms/step - loss: 0.4920 - accuracy: 0.7672 - val_loss: 0.5422 - val_accuracy: 0.7382
Epoch 9/20
4500/4500 [==============================] - 12s 3ms/step - loss: 0.4911 - accuracy: 0.7685 - val_loss: 0.5728 - val_accuracy: 0.7356
Epoch 10/20
4500/4500 [==============================] - 12s 3ms/step - loss: 0.4880 - accuracy: 0.7715 - val_loss: 0.5691 - val_accuracy: 0.7384
Epoch 11/20
4500/4500 [==============================] - 12s 3ms/step - loss: 0.4886 - accuracy: 0.7733 - val_loss: 0.5569 - val_accuracy: 0.7364
Epoch 12/20
4500/4500 [==============================] - 12s 3ms/step - loss: 0.4868 - accuracy: 0.7734 - val_loss: 0.5645 - val_accuracy: 0.7279
Epoch 13/20
4500/4500 [==============================] - 12s 3ms/step - loss: 0.4861 - accuracy: 0.7740 - val_loss: 0.6080 - val_accuracy: 0.7356
Epoch 14/20
4500/4500 [==============================] - 12s 3ms/step - loss: 0.4816 - accuracy: 0.7765 - val_loss: 0.5894 - val_accuracy: 0.7321
Epoch 15/20
4500/4500 [==============================] - 12s 3ms/step - loss: 0.4846 - accuracy: 0.7779 - val_loss: 0.6403 - val_accuracy: 0.7274
Epoch 16/20
4500/4500 [==============================] - 12s 3ms/step - loss: 0.4820 - accuracy: 0.7779 - val_loss: 0.5974 - val_accuracy: 0.7352
Epoch 17/20
4500/4500 [==============================] - 12s 3ms/step - loss: 0.4831 - accuracy: 0.7796 - val_loss: 0.6507 - val_accuracy: 0.7274
Epoch 18/20
4500/4500 [==============================] - 12s 3ms/step - loss: 0.4833 - accuracy: 0.7791 - val_loss: 0.6883 - val_accuracy: 0.7327
Epoch 19/20
4500/4500 [==============================] - 12s 3ms/step - loss: 0.4837 - accuracy: 0.7790 - val_loss: 0.7188 - val_accuracy: 0.7312
Epoch 20/20
4500/4500 [==============================] - 12s 3ms/step - loss: 0.4803 - accuracy: 0.7790 - val_loss: 0.6159 - val_accuracy: 0.7349

Always cannot overfit. Is there any way to reduce the loss?

2

The hint on the number layers is a good place to start for the execise markdown. You don’t need a NN with 5 layers (2 conv + 1 maxpool + 1 globalmaxpool + 1 dense). A smaller network is sufficient for this assignment.
Do remember that the choice of optimizer is also important for a NN to be effective.

Please use learning_rate and not lr since the later is deprecated.