Dinosaurs model W1A2 -- model

Course Q&A Deep Learning Specialization Sequence Models
1

Hi

I am having the following issue:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-20-725c093d6b91> in <module>
----> 1 parameters, last_name = model(data.split("\n"), ix_to_char, char_to_ix, 22001, verbose = True)
      2 
      3 assert last_name == 'Trodonosaurus\n', "Wrong expected output"
      4 print("\033[92mAll tests passed!")

<ipython-input-19-37ecc67f3dc9> in model(data_x, ix_to_char, char_to_ix, num_iterations, n_a, dino_names, vocab_size, verbose)
     62         # Perform one optimization step: Forward-prop -> Backward-prop -> Clip -> Update parameters
     63         # Choose a learning rate of 0.01
---> 64         curr_loss, gradients, a_prev = optimize(X, Y, a_prev, parameters, learning_rate=0.01)
     65 
     66         ### END CODE HERE ###

<ipython-input-9-fca0aba78113> in optimize(X, Y, a_prev, parameters, learning_rate)
     32 
     33     # Forward propagate through time (≈1 line)
---> 34     loss, cache = rnn_forward(X, Y, a_prev, parameters)
     35 
     36     # Backpropagate through time (≈1 line)

~/work/W1A2/utils.py in rnn_forward(X, Y, a0, parameters, vocab_size)
    100 
    101         # Update the loss by substracting the cross-entropy term of this time-step from it.
--> 102         loss -= np.log(y_hat[t][Y[t],0])
    103 
    104     cache = (y_hat, a, x)

TypeError: 'int' object is not subscriptable

Is it possible the implementation of rnn_forward to be incorrect?

2

A perfectly correct function can still throw errors if you pass it wrong arguments. Given that code was provided for you, it’s a bad idea to assume any error thrown by that function is “somebody else’s fault” :grin:. If the provided code was broken, you can reasonably assume we would have heard about it by now. E.g. it worked ok for me. :nerd_face:

My guess is that somehow the y_hat value in that expression is missing a dimension or is the wrong type. So the thing to look for would be wrong shapes for some of the parameter values or the like.

3

Yeah. I forgot it was your implementation and I thought it was mine. :sweat_smile:

Thank you. :slight_smile:

4

How does the y_hat affect my issue when it hasn’t been used in my function?

Update: I forgot to initiate the Y correctly, but now I have a wrong output. :confused:

AssertionError                            Traceback (most recent call last)
<ipython-input-26-725c093d6b91> in <module>
      1 parameters, last_name = model(data.split("\n"), ix_to_char, char_to_ix, 22001, verbose = True)
      2 
----> 3 assert last_name == 'Trodonosaurus\n', "Wrong expected output"
      4 print("\033[92mAll tests passed!")

AssertionError: Wrong expected output
5

This is how debugging works: you have to start from the point of the error and understand what is wrong at that point. Then you have to track backwards to figure out how it happened. The dimension of y_hat is wrong down in rnn_forward: it looks like it’s just an integer, when it is expected to be an array of integers. You can start by looking at the code in rnn_forward to see how y_hat is generated. Click “File → Open” and then open the python file with the utility functions. You can deduce the name of the file by examining the “Import” cell early in the notebook.

That path will lead you back to which variable that you passed to rnn_forward that is wrong and is causing y_hat to be wrong.

There is also a “forward” way of doing debugging. Start with the code you are writing and figure out what the dimensions of all the input objects are. From that, you then compute what the shapes of the generated objects should be. What are the arguments that are passed to rnn_forward? One of them (at least) must be wrong …

6

What is the actual output you got that is wrong?

One common way to a different answer is to use the raw inputs, instead of the randomly scrambled version that they create for you in the logic they give you.

7

Although I have incredible low Loss, I have definitely just random words as output.
I am thinking that the problem is behind Y, because the newline character isn’t the last letter in the vector. Am I right?

j =  0 idx =  0
single_example = turiasaurus
single_example_chars [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26]
single_example_ix ['\n', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z']
 X =  [None, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26] 
 Y =        [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 0]
8

The single_example_chars are not supposed to be the entire alphabet - they should just be the representation of the current example.

And they should be the characters, not the indices.

The single_example_ix is the index values. You look them up from the char_to_ix dictionary.

9

I think that I use the correct dictionaries, but maybe my list comprehension implementation is the cause. Can you look it because I am not sure what I am doing wrong?

single_example_chars equals to a list that has char for char in single_example
single_example_ix is a list with single_example_chars for single_example_chars in char_to_ix
X = [None] + single_example_chars

10

But where is the actual input value in that code? You’re just enumerating the dictionaries themselves. So the answer will always be the same, right? That can’t be what you want. Also note that enumerating a dictionary may give you the keys instead of the answers. It looks like that’s what happened in your case: the results are “backwards” in terms of the actual type of the values, right?

Update: you changed your post after I wrote the above reply. You’re right that single_example_chars is a list with the actual characters in the current sample. And single_example_idx is a list with the corresponding index values. Is that what you get from your code? But also note that X is a list of index values, not character values, right?

11

But in the notes it is mentioned Create the list of input characters: X
single_example_idx doesn’t map the characters with indexes in my code :

    `single_example_ix = [char_to_ix for char_to_ix in single_example_chars]`

Output from print: single_example_ix [‘t’, ‘u’, ‘r’, ‘i’, ‘a’, ‘s’, ‘a’, ‘u’, ‘r’, ‘u’, ‘s’]

12

Sorry, but that is wrong. That is a problem. In the instructions, sometimes they are sloppy and say “character”, but mean that it would be expressed in that context as an index. You have two choices at any point, right? When we’re doing computations, we need the numeric index representations and we only need the actual letters when we want to show the results.

You need to fix your logic so that single_example_ix and X are expressed as index values.

13

That should probably say “Create X as a list of character indices”.