Emojify exercise 4 embedding layer

Course Q&A Deep Learning Specialization Sequence Models
1

I don’t know embedding layer input/output dimension. Help me please.
It says that no matter what the value is, it’s wrong.
My code is here,

emb_matrix = np.zeros((vocab_size,1))

for word, idx in word_to_index.items():
    emb_matrix[idx, :] = word_to_index[word]

embedding_layer = Embedding(vocab_size, word_to_vec_map[any_word].shape[0])

I’ve been thinking about in for 3 days.

2

Hi @dragonix,

The dimensions of Your emb_maxtrix is incorrect. Here is the implementation notes:
Implement pretrained_embedding_layer() with these steps:

  1. Initialize the embedding matrix as a numpy array of zeros.
  • The embedding matrix has a row for each unique word in the vocabulary.
    • There is one additional row to handle “unknown” words.
    • So vocab_size is the number of unique words plus one.
  • Each row will store the vector representation of one word.
    • For example, one row may be 50 positions long if using GloVe word vectors.
  • In the code below, emb_dim represents the length of a word embedding.

so
em_matrix = np.zeros((vocab_size, emb_dim))

3

Hi, Kic
Thank you for your reply.
I’ve already tried what you said.
emb_dim = 2, is this correct?
And emb_matrix =
[[ 0. 0.]
[ 1. 1.]
[ 2. 2.]
[ 3. 3.]
[ 4. 4.]
[ 5. 5.]
[ 6. 6.]
[ 7. 7.]
[ 8. 8.]
[ 9. 9.]
[10. 10.]
[11. 11.]
[12. 12.]
[13. 13.]
[ 0. 0.]]
I see the error message below, but it’s the same even if I try this and that.
I think the Embedding() part is wrong.

AssertionError Traceback (most recent call last)
in
28
29
—> 30 pretrained_embedding_layer_test(pretrained_embedding_layer)

in pretrained_embedding_layer_test(target)
24 [[[ 3, 3], [ 3, 3], [ 2, 4], [ 3, 2], [ 3, 4],
25 [-2, 1], [-2, 2], [-1, 2], [-1, 1], [-1, 0],
—> 26 [-2, 0], [-3, 0], [-3, 1], [-3, 2], [ 0, 0]]]), “Wrong vaulues”
27 print("\033[92mAll tests passed!")
28

AssertionError: Wrong vaulues

4

Hi @dragonix ,

The instruction for step3:

# Define Keras embedding layer with the correct input and output sizes
# Make it non-trainable.

Your input and output for the embedding layer is correct, but there is one parameter ‘trainable’ is missing, it should be set to trainable=False

5

Hi @Kic
Thank you for your quick reply.
But the result was the same.
I changed ‘word_to_index’ to ‘word_to_vec_map’ in Step 2, so I passed. Thank you~

6

Hi, @Kic
There’s another problem.
What’s the problem?

embedding_layer = pretrained_embedding_layer(word_to_vec_map, word_to_index)
print(“weights[0][1][1] =”, embedding_layer.get_weights()[0][1][1])
print(“Input_dim”, embedding_layer.input_dim)
print(“Output_dim”,embedding_layer.output_dim)

KeyError Traceback (most recent call last)
in
----> 1 embedding_layer = pretrained_embedding_layer(word_to_vec_map, word_to_index)
2 print(“weights[0][1][1] =”, embedding_layer.get_weights()[0][1][1])
3 print(“Input_dim”, embedding_layer.input_dim)
4 print(“Output_dim”,embedding_layer.output_dim)

in pretrained_embedding_layer(word_to_vec_map, word_to_index)
29 for word, idx in word_to_index.items():
30 #emb_matrix[idx, :] = word_to_index[word]
—> 31 emb_matrix[idx, :] = word_to_vec_map[word]
32
33

KeyError: ‘´0.=’

7

Hi @dragonix ,

I couldn’t see any problem with the logic in that section of the code. It may help to refresh your kernel and clear all output, rerun the cells from start to ensure your code is executed in a clean environment.

8

Hi @Kic

Even if I do what you say, the result is the same.
And ‘AttributeError’ also occurs in 2.5 - Train the Model

Anyway, I passed all the exercises 1-5 and my score is 0. The message below comes out.
I’d appreciate it if you could help me.

Cell #27. Can’t compile the student’s code. Error: KeyError(‘i’,)

9

Hi @dragonix,

Could you post that section of the code in a DM, I’ll have a look for you.

10

Hi @Kic ,
It is the same cell as the posting above.
I think exercise 4 is the problem.

emb_matrix = np.zeros((vocab_size, emb_dim))

for word, idx in word_to_index.items():
emb_matrix[idx, ] = word_to_vec_map[word]

embedding_layer = Embedding(vocab_size, emb_dim, trainable=False)

KeyError Traceback (most recent call last)
in
----> 1 embedding_layer = pretrained_embedding_layer(word_to_vec_map, word_to_index)
2 print(“weights[0][1][1] =”, embedding_layer.get_weights()[0][1][1])
3 print(“Input_dim”, embedding_layer.input_dim)
4 print(“Output_dim”,embedding_layer.output_dim)

in pretrained_embedding_layer(word_to_vec_map, word_to_index)
29 for word, idx in word_to_index.items():
30 #emb_matrix[idx, :] = word_to_vec_map[word]
—> 31 emb_matrix[idx, ] = word_to_vec_map[word]
32
33 # Step 3

KeyError: ‘´0.=’

11

Hi @dragonix ,

The index/slicing to the emb_matrix is incorrect, to get the whole vector stored in emb_matrix at row pointed to by idx, it shoudl be:
emb_matrix[idx, :] = word_to_vec_map[word]

12

Hi @Kic
Unfortunately, it is the same result again.
I don’t know what the hell to do
Cell #27. Can’t compile the student’s code. Error: KeyError(‘i’,)

<Cell#27>
embedding_layer = pretrained_embedding_layer(word_to_vec_map, word_to_index)
print(“weights[0][1][1] =”, embedding_layer.get_weights()[0][1][1])
print(“Input_dim”, embedding_layer.input_dim)
print(“Output_dim”,embedding_layer.output_dim)

KeyError Traceback (most recent call last)
in
----> 1 embedding_layer = pretrained_embedding_layer(word_to_vec_map, word_to_index)
2 print(“weights[0][1][1] =”, embedding_layer.get_weights()[0][1][1])
3 print(“Input_dim”, embedding_layer.input_dim)
4 print(“Output_dim”,embedding_layer.output_dim)

in pretrained_embedding_layer(word_to_vec_map, word_to_index)
29 for word, idx in word_to_index.items():
30 #print(word, idx)
—> 31 emb_matrix[idx,:] = word_to_vec_map[word]
32
33 #print(emb_matrix.shape)

KeyError: ‘´0.=’

<Cell #30>
model = Emojify_V2((maxLen,), word_to_vec_map, word_to_index)
model.summary()

(None, 10)

KeyError Traceback (most recent call last)
in
----> 1 model = Emojify_V2((maxLen,), word_to_vec_map, word_to_index)
2 model.summary()

in Emojify_V2(input_shape, word_to_vec_map, word_to_index)
22 print(sentence_indices.shape)
23 # Create the embedding layer pretrained with GloVe Vectors (≈1 line)
—> 24 embedding_layer = pretrained_embedding_layer(word_to_vec_map, word_to_index)
25
26 # Propagate sentence_indices through your embedding layer

in pretrained_embedding_layer(word_to_vec_map, word_to_index)
29 for word, idx in word_to_index.items():
30 #print(word, idx)
—> 31 emb_matrix[idx,:] = word_to_vec_map[word]
32
33 #print(emb_matrix.shape)

KeyError: ‘´0.=’

13

Hi @dragonix ,

From your post, your code is executing
model = Emojify_V2((maxLen,), word_to_vec_map, word_to_index)
How could that happen? There are lots of test cases before that call, have all the tests been passed?
The changes I suggested should work, I have done it and tested myself. Try refreshing the kernel and rerun all the cells up to and including this cell:

embedding_layer = pretrained_embedding_layer(word_to_vec_map, word_to_index)
print(“weights[0][1][1] =”, embedding_layer.get_weights()[0][1][1])
print(“Input_dim”, embedding_layer.input_dim)
print(“Output_dim”,embedding_layer.output_dim)

14

Hi, @Kic
It’s refreshing, so it works well.
It’s all thanks to you. Thank you.

15

Hi @dragonix ,

That is great.
I was checking your code, it looked fine to me. I was about to send you a message, but you are ahead. Great news.