Even though I used different layer’s in Bidirectional(LSTM()),I went till 2048 but maximum accuracy I obtained is Ranging between 60% to 67%
Please share the following details:
- The values of units did use for LSTM.
- Size of the embedding dimension.
Sir, I used 128,256,512,1024,2048 and the Embedding dimensions are input_dim=total_words,output_dim=100, input_length=max_sequence_len-1
Please click my name and message your notebook as an attachment.
The learning rate of 1e-2 is high for the optimizer for your training setup. A higher learning rate does not mean that model weights will converge quicker. While it’s true that weight updates will involve larger steps, the performance is affected by oscillations around optimal weights.
The default learning rate of rmsprop is 1e-3. When tuning learning rate, it’s important to experiment with candidate values and see how the model loss changes for different values. When in doubt, start with the default learning rate and tune your model architecture.
Try even other optimizers like adam to train your model as well.