Hi
I tried a lot of combinations but my model always get to be overfitting.
What is wrong with my model
Ex:
model = tf.keras.Sequential([
tf.keras.layers.Embedding(vocab_size+1, embedding_dim, input_length=maxlen, weights=[embeddings_matrix], trainable=False),
tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(64, return_sequences=True)),
tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(64, dropout=.2)),
tf.keras.layers.Dense(128, activation=‘relu’),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(64, activation=‘relu’),
tf.keras.layers.Dense(1, activation=‘sigmoid’)])
model.compile(loss=tf.losses.BinaryCrossentropy(),
optimizer=tf.keras.optimizers.RMSprop(),
metrics=['accuracy'])
Epoch 1/20
4500/4500 [==============================] - 212s 46ms/step - loss: 0.5429 - accuracy: 0.7228 - val_loss: 0.4987 - val_accuracy: 0.7565
Epoch 2/20
4500/4500 [==============================] - 204s 45ms/step - loss: 0.4918 - accuracy: 0.7620 - val_loss: 0.4803 - val_accuracy: 0.7667
Epoch 3/20
4500/4500 [==============================] - 206s 46ms/step - loss: 0.4686 - accuracy: 0.7781 - val_loss: 0.4990 - val_accuracy: 0.7542
Epoch 4/20
4500/4500 [==============================] - 206s 46ms/step - loss: 0.4514 - accuracy: 0.7887 - val_loss: 0.4712 - val_accuracy: 0.7750
Epoch 5/20
4500/4500 [==============================] - 207s 46ms/step - loss: 0.4375 - accuracy: 0.7976 - val_loss: 0.4771 - val_accuracy: 0.7805
Epoch 6/20
4500/4500 [==============================] - 206s 46ms/step - loss: 0.4242 - accuracy: 0.8048 - val_loss: 0.4747 - val_accuracy: 0.7799
Epoch 7/20
4500/4500 [==============================] - 206s 46ms/step - loss: 0.4108 - accuracy: 0.8127 - val_loss: 0.4924 - val_accuracy: 0.7769
Epoch 8/20
4500/4500 [==============================] - 208s 46ms/step - loss: 0.3997 - accuracy: 0.8204 - val_loss: 0.5116 - val_accuracy: 0.7738
Epoch 9/20
4500/4500 [==============================] - 209s 46ms/step - loss: 0.3880 - accuracy: 0.8274 - val_loss: 0.5173 - val_accuracy: 0.7769
Epoch 10/20
4500/4500 [==============================] - 207s 46ms/step - loss: 0.3768 - accuracy: 0.8336 - val_loss: 0.4914 - val_accuracy: 0.7730
Epoch 11/20
4500/4500 [==============================] - 210s 47ms/step - loss: 0.3663 - accuracy: 0.8387 - val_loss: 0.5730 - val_accuracy: 0.7753
Epoch 12/20
4500/4500 [==============================] - 210s 47ms/step - loss: 0.3581 - accuracy: 0.8436 - val_loss: 0.5930 - val_accuracy: 0.7706
Epoch 13/20
4500/4500 [==============================] - 206s 46ms/step - loss: 0.3493 - accuracy: 0.8484 - val_loss: 0.5829 - val_accuracy: 0.7713
Epoch 14/20
4500/4500 [==============================] - 206s 46ms/step - loss: 0.3409 - accuracy: 0.8535 - val_loss: 0.5440 - val_accuracy: 0.7661
Epoch 15/20
4500/4500 [==============================] - 203s 45ms/step - loss: 0.3327 - accuracy: 0.8576 - val_loss: 0.5678 - val_accuracy: 0.7681
Epoch 16/20
4500/4500 [==============================] - 203s 45ms/step - loss: 0.3260 - accuracy: 0.8612 - val_loss: 0.6521 - val_accuracy: 0.7676
Epoch 17/20
4500/4500 [==============================] - 206s 46ms/step - loss: 0.3183 - accuracy: 0.8660 - val_loss: 0.7046 - val_accuracy: 0.7668
Epoch 18/20
4500/4500 [==============================] - 206s 46ms/step - loss: 0.3102 - accuracy: 0.8709 - val_loss: 0.7240 - val_accuracy: 0.7621
Epoch 19/20
4500/4500 [==============================] - 209s 46ms/step - loss: 0.3043 - accuracy: 0.8730 - val_loss: 0.7048 - val_accuracy: 0.7600
Epoch 20/20
4500/4500 [==============================] - 207s 46ms/step - loss: 0.2979 - accuracy: 0.8766 - val_loss: 0.5961 - val_accuracy: 0.7574
The slope of your validation loss curve is 0.01193
- List item
Enjoy the rest of the course. 