I think this topics is no longer the one to be discussed in this community… 
So, here is the last advice. I quickly did it, and the result was,
(TensorSpec(shape=(96, 128, 3), dtype=tf.float32, name=None), TensorSpec(shape=(96, 128, 1), dtype=tf.uint8, name=None))
Epoch 1/40
34/34 [==============================] - 39s 425ms/step - loss: 1.9192 - accuracy: 0.3911
Epoch 2/40
34/34 [==============================] - 5s 134ms/step - loss: 1.0832 - accuracy: 0.7269
Epoch 3/40
34/34 [==============================] - 5s 134ms/step - loss: 0.6486 - accuracy: 0.8212
Epoch 4/40
34/34 [==============================] - 5s 135ms/step - loss: 0.5192 - accuracy: 0.8484
Epoch 5/40
34/34 [==============================] - 5s 135ms/step - loss: 0.4509 - accuracy: 0.8690
Epoch 6/40
34/34 [==============================] - 5s 136ms/step - loss: 0.3706 - accuracy: 0.8893
Epoch 7/40
34/34 [==============================] - 5s 136ms/step - loss: 0.3306 - accuracy: 0.9014
Epoch 8/40
34/34 [==============================] - 5s 138ms/step - loss: 0.2851 - accuracy: 0.9150
Epoch 9/40
34/34 [==============================] - 5s 138ms/step - loss: 0.2500 - accuracy: 0.9261
Epoch 10/40
34/34 [==============================] - 5s 138ms/step - loss: 0.2181 - accuracy: 0.9353
Epoch 11/40
34/34 [==============================] - 5s 139ms/step - loss: 0.2738 - accuracy: 0.9188
Epoch 12/40
34/34 [==============================] - 5s 139ms/step - loss: 0.2252 - accuracy: 0.9326
Epoch 13/40
34/34 [==============================] - 5s 140ms/step - loss: 0.1856 - accuracy: 0.9443
Epoch 14/40
34/34 [==============================] - 5s 141ms/step - loss: 0.1708 - accuracy: 0.9486
Epoch 15/40
34/34 [==============================] - 5s 141ms/step - loss: 0.1674 - accuracy: 0.9497
Epoch 16/40
34/34 [==============================] - 5s 142ms/step - loss: 0.1508 - accuracy: 0.9549
Epoch 17/40
34/34 [==============================] - 5s 143ms/step - loss: 0.1440 - accuracy: 0.9568
Epoch 18/40
34/34 [==============================] - 5s 143ms/step - loss: 0.1339 - accuracy: 0.9599
Epoch 19/40
34/34 [==============================] - 5s 143ms/step - loss: 0.1245 - accuracy: 0.9627
Epoch 20/40
34/34 [==============================] - 5s 144ms/step - loss: 0.1202 - accuracy: 0.9639
Epoch 21/40
34/34 [==============================] - 5s 143ms/step - loss: 0.1137 - accuracy: 0.9660
Epoch 22/40
34/34 [==============================] - 5s 143ms/step - loss: 0.1084 - accuracy: 0.9672
Epoch 23/40
34/34 [==============================] - 5s 142ms/step - loss: 0.1032 - accuracy: 0.9688
Epoch 24/40
34/34 [==============================] - 5s 142ms/step - loss: 0.0998 - accuracy: 0.9698
Epoch 25/40
34/34 [==============================] - 5s 141ms/step - loss: 0.0948 - accuracy: 0.9714
Epoch 26/40
34/34 [==============================] - 5s 141ms/step - loss: 0.0928 - accuracy: 0.9716
Epoch 27/40
34/34 [==============================] - 5s 141ms/step - loss: 0.0908 - accuracy: 0.9719
Epoch 28/40
34/34 [==============================] - 5s 142ms/step - loss: 0.0904 - accuracy: 0.9721
Epoch 29/40
34/34 [==============================] - 5s 141ms/step - loss: 0.0839 - accuracy: 0.9742
Epoch 30/40
34/34 [==============================] - 5s 141ms/step - loss: 0.0843 - accuracy: 0.9741
Epoch 31/40
34/34 [==============================] - 5s 142ms/step - loss: 0.0802 - accuracy: 0.9751
Epoch 32/40
34/34 [==============================] - 5s 142ms/step - loss: 0.0796 - accuracy: 0.9752
Epoch 33/40
34/34 [==============================] - 5s 143ms/step - loss: 0.0730 - accuracy: 0.9774
Epoch 34/40
34/34 [==============================] - 5s 143ms/step - loss: 0.0729 - accuracy: 0.9771
Epoch 35/40
34/34 [==============================] - 5s 143ms/step - loss: 0.0719 - accuracy: 0.9774
Epoch 36/40
34/34 [==============================] - 5s 143ms/step - loss: 0.0713 - accuracy: 0.9776
Epoch 37/40
34/34 [==============================] - 5s 143ms/step - loss: 0.0698 - accuracy: 0.9779
Epoch 38/40
34/34 [==============================] - 5s 143ms/step - loss: 0.0666 - accuracy: 0.9790
Epoch 39/40
34/34 [==============================] - 5s 143ms/step - loss: 0.0653 - accuracy: 0.9794
Epoch 40/40
34/34 [==============================] - 5s 142ms/step - loss: 0.0649 - accuracy: 0.9794
So, … please enjoy your debugging !