C3W4 assignment accuracy lower than 80%

Course Q&A TensorFlow Developer Professional Natural Language Processing in TensorFlow
1

Hi, All,
I have tried increased the lstm_units from 64 to 500, and also tried to add multiple layers of lstm, still can not reach 80%. Please help with any suggestions.

Thanks

2

There are a bunch of posts on ways to tune a NN to achieve the required level of performance. Please start with this post and explore options.

3

I have tried so far----
lstm_units —Accuracy
90 – 0.71
100–0.7293
128 – 0.7371/0.7401
140-- 0.7129/0.7235
150 – 0.7425
160-- 0.7267
175–0.7222
180–0.70
200–0.7262/0.7223
256–0.7188
300–0.69
400–0.67
500–0.50
512–0.6667
600–0.6571
800–0.6273
1024–0.56

Then with 128 lstm_units for first layer, add one more layer of BRNN with 32 lstm_units, accuracy drop to 0.14

If set 1st layer as Conv1D, the accuracy will drop < 0.1.

For 3 days, I struck here …

4

Clearly, you haven’t read the links in the post I gave you:

The number of units in a dense layer / the number of nodes in a dense layer are usually powers of 2 (a heurestic that can be observed in many models).

Follow the unit count for lstm and other layers as well.

To me, setting the 1st layer as Conv1D doesn’t make sense since this is a text related assignment and there’s no point in the Conv layer interpreting integer encodings of words.

5

Thanks so much for the quick response!

I have tried the following:
One layer BRNN with different lstm_units
lstm_units --Accuracy
32 – 0.6121
64–0.6784
128–0.7401
256–0.7188
512–0.667
1024 --0.56
One layer Brnn + Lstm
64 + 32 --0.2687
64+64–0.4414
128+32–0.1289
128+64–0.1215
128+128–0.0723
256+32–0.0611
256+64–0.0961
256+128–0.0330
One layer BRNN + Brnn
128+32-0.0805
128+64–0.5332
128+128–0.0490
256+32–0.0342
256+64–0.0528
256+128–0.0312

Still no good so far…

6

Please click my name and message your notebook as an attachment.

7

Here are some hints:

  1. Set the number of embeddings to a high value < 1024 but a power of 2.
  2. Let the number of LSTM units in the model increase as you go deeper in the network.

I was able to achieve ~84.9 % accuracy just by changing the number of units and the embedding layer dimension.

9

Please do 2 more things:

  1. Leave the learning rate at the default value for the optimizer you’ve picked. Your learning rate is high.
  2. Start with a smaller architecture before adding additional layers. That’ll give you a good place to start exploring paths. Other than the output layer, keep your nodes in a layer < 1024.
10

Thanks so much, Balaji!

After 5 days for this assignment, it finally passed the 80%.