Course 5 week1 dinosaur model, loss exploding after 5 iterations

Course Q&A Deep Learning Specialization Sequence Models
1

Hi, The model seems running but the loss is exploding after few iterations. Here are the outputs:
single_example = turiasaurus
single_example_chars [‘t’, ‘u’, ‘r’, ‘i’, ‘a’, ‘s’, ‘a’, ‘u’, ‘r’, ‘u’, ‘s’]
single_example_ix [20, 21, 18, 9, 1, 19, 1, 21, 18, 21, 19]
X = [None, 20, 21, 18, 9, 1, 19, 1, 21, 18, 21, 19]
Y = [20, 21, 18, 9, 1, 19, 1, 21, 18, 21, 19, 0]
…

curr loss= 58.12372726116307
curr loss= 50.29752492287171
curr loss= 151.48395730840713
curr loss= 1566.987862638166
curr loss= 4024.7637708785633
curr loss= inf
curr loss= inf

The call to the optimizer seems OK:
curr_loss, gradients, a_prev = optimize(X, Y, a_prev, parameters, learning_rate = 0.01)

Can anyone help me on this topic ?
many thanks in advance
Marc

2

Hi,
Even if this particular assignment has already been graded with success (but only up to 75/100), I would be pleased to know where my code fails. Who of the staff could I send my code to in order to understand my error ?
Thanks
Marc

3

Hi,
Could you DM me your code? Thanks.

4

Hi Edward,

Thank you for your prompt answer. Please find attached my code.

Best regards

Marc

Scarica Outlook per Android

(Attachment Dinosaurus_Island_Character_level_language_model.json is missing)

5

Me too, ran into that.
It turns out your optimize() function updates parameters to the wrong directions of gradient (that is not checked in tests)
Revert the sign and you’ll be good.

6

Thank you Edward and Gleb for your support, now it works fine :slight_smile:

7

Hi, My problem is the opposite. Instead of exploding, loss seems to drop very quickly. All tests pass before exercise 4. The algorithm starts fine, then drops in loss immediately. Is there any place to me to, so I can see what I’m doing wrong? I have reloaded the Workspace twice, tried rebooting the server, restarted kernel and and cleared output, but same problem.

j = 0 idx = 0
single_example = turiasaurus
single_example_chars [‘t’, ‘u’, ‘r’, ‘i’, ‘a’, ‘s’, ‘a’, ‘u’, ‘r’, ‘u’, ‘s’]
single_example_ix [20, 21, 18, 9, 1, 19, 1, 21, 18, 21, 19]
X = [None, 20, 21, 18, 9, 1, 19, 1, 21, 18, 21, 19]
Y = [20, 21, 18, 9, 1, 19, 1, 21, 18, 21, 19, 0]

Iteration: 0, Loss: 23.087336

Nkzxwtdmfqoeyhsqwasjkjvu
Kneb
Kzxwtdmfqoeyhsqwasjkjvu
Neb
Zxwtdmfqoeyhsqwasjkjvu
Eb
Xwtdmfqoeyhsqwasjkjvu

j = 1535 idx = 2
j = 1536 idx = 3
Iteration: 2000, Loss: 9.894264

Iavosaurus
Dosaurus
Esitosaurus
Iaecerus
Unus
Amanonelhoneris
Tosaurus

…

Iteration: 22000, Loss: 0.294620

Iavesaqr
Esitoriasaurus
Esitoriasaurus
Iaeaurus
Urus
Andoravenator
Saurur

Thanks, Mark N.

8

@Mhmemeth, your post is a duplicate, I replied on your other thread.