Wk2 doubt about dimensions specified in Embedding()

Course Q&A Deep Learning Specialization Sequence Models
1

Hi,

I am not sure about the input and output dimensions that should be supplied to the Embedded().
From the hint, it seems the input_dim = vocab_size, but I am not sure about the output_dim. I am not clear what is meant by “number of positions in a word embedding”. Could you please help.

If I specify output_dim = len(word_to_vec_map), I get the following error:
ValueError: Layer weight shape (15, 14) not compatible with provided weight shape (15, 16)

Thanks,
Suprio

2

Which Assignment number?

3

Hi @suprio,

The output dim for the embedding layer should be emb_dim.

4

Hi, Kic,

Thanks for your response.
I am doing this: embedding_layer = Embedding(input_dim = vocab_size,output_dim = emb_dim ,trainable=False)
But still seeing the error:

ValueError: Layer weight shape (15, 2) not compatible with provided weight shape (15, 16)

Thanks,
Suprio

5

Hi @suprio ,

Could you post any error track here to give us more info to work with.

6

Here you go! Thanks a bunch!

ValueError Traceback (most recent call last)
in
28
29
—> 30 pretrained_embedding_layer_test(pretrained_embedding_layer)

in pretrained_embedding_layer_test(target)
16
17 np.random.seed(1)
—> 18 embedding_layer = target(word_to_vec_map, word_to_index)
19
20 assert type(embedding_layer) == Embedding, “Wrong type”

in pretrained_embedding_layer(word_to_vec_map, word_to_index)
41
42 # Set the weights of the embedding layer to the embedding matrix. Your layer is now pretrained.
—> 43 embedding_layer.set_weights([emb_matrix])
44
45 return embedding_layer

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py in set_weights(self, weights)
1824 raise ValueError(
1825 'Layer weight shape %s not compatible with provided weight ’
→ 1826 'shape s' (ref_shape, weight.shape))
1827 weight_value_tuples.append((param, weight))
1828 weight_index += 1

ValueError: Layer weight shape (15, 2) not compatible with provided weight shape (15, 16)

7

Hi @suprio ,

print out the shape of emb_matrix to find out what it is, that might give a clue of where the problem lies.

8

Hi, Kic,

Thanks much for helping with my queries.
So emb_matrix seems to have dimension 15x14 (vocab_size x length of word_to_vec_map).
However, I am still getting the following output shape error. Could you please give some more clue?

AssertionError Traceback (most recent call last)
in
28
29
—> 30 pretrained_embedding_layer_test(pretrained_embedding_layer)

in pretrained_embedding_layer_test(target)
20 assert type(embedding_layer) == Embedding, “Wrong type”
21 assert embedding_layer.input_dim == len(list(word_to_vec_map.keys()))+1, “Wrong input shape”
—> 22 assert embedding_layer.output_dim == len(word_to_vec_map[‘a’]), “Wrong output shape”
23 assert np.allclose(embedding_layer.get_weights(),
24 [[[ 3, 3], [ 3, 3], [ 2, 4], [ 3, 2], [ 3, 4],

AssertionError: Wrong output shape

Thanks,
Suprio

9

i think emb_matrix shape should be vocab_Size*emb_dim

10

Hi @suprio , @ADITYA_AGARWAL

The shape of the emb_matrix is (vocab_size, emb_dim). Please check step 1 of the implementation notes.

1 Like
11

Thanks Kic!
I had specified that too, and seeing this error:

AssertionError Traceback (most recent call last)
in
28
29
—> 30 pretrained_embedding_layer_test(pretrained_embedding_layer)

in pretrained_embedding_layer_test(target)
24 [[[ 3, 3], [ 3, 3], [ 2, 4], [ 3, 2], [ 3, 4],
25 [-2, 1], [-2, 2], [-1, 2], [-1, 1], [-1, 0],
—> 26 [-2, 0], [-3, 0], [-3, 1], [-3, 2], [ 0, 0]]]), “Wrong vaulues”
27 print("\033[92mAll tests passed!")
28

AssertionError: Wrong vaulues

12

Hi @suprio ,

If your emb_matrix has the correct dimension, ie. shape, there should not be any problem. The assertion is triggered because the weights of the emb_matrix is different from these
[[[ 3, 3], [ 3, 3], [ 2, 4], [ 3, 2], [ 3, 4],
[-2, 1], [-2, 2], [-1, 2], [-1, 1], [-1, 0],
[-2, 0], [-3, 0], [-3, 1], [-3, 2], [ 0, 0]]]

print out what the values are returned from calling embedding_layer.get_weights(), that would give us some idea as to what is happening.
Also, just to make sure you have a clean execution environment, fresh the kernel and rerun the code cells from start.

13

This is the output of embedded_layer.get_weight() :
[array([[ 0., 0.],
[ 1., 1.],
[ 2., 2.],
[ 3., 3.],
[ 4., 4.],
[ 5., 5.],
[ 6., 6.],
[ 7., 7.],
[ 8., 8.],
[ 9., 9.],
[10., 10.],
[11., 11.],
[12., 12.],
[13., 13.],
[ 0., 0.]], dtype=float32)]

This comes after I started afresh.

Thanks much,
Suprio

14

Hi @suprio ,

It looks like there is a problem with your code for step 2. If you put that function in DM to me, I will have a look for you.

1 Like
15

This Solves another error which i was having, Error was : “Too many indices”