C4_W2 Text Summary Assignment: Model outputting only [SOS]

Course Q&A Natural Language Processing NLP with Attention Models
1

Hello,

Need some help to figure out why the model is only outputting [SOS] .

Model output after training for 20 Epoches:

Epoch 20, Loss 3.877531
Time taken for one epoch: 8.62150263786316 sec
Example summarization on the test set:
  True summarization:
    [SOS] hannah needs betty's number but amanda doesn't have it. she needs to contact larry. [EOS]
  Predicted summarization:
    [SOS]

When I run the next_word function the output for the untrained model returns an empty list instead of the expected output mentioned in the notebook. However all unit tests are passing.

Predicted token: []
Predicted word:
2

I think I’ve resolved the issue with next word and now get the model’s next word output along with a summary. I printed the output of create_look_ahead_mask and worked backwards to verify it’s being constructed correctly for the sequence length.

1 Like