Input dimension for Embedding Layer Week1 Assignment

Course Q&A Natural Language Processing NLP with Attention Models
1

I thought if we use mask_zero = True then input_dim = vocab_size+1 as the documentation says and also we did that in the previous week’s assignment as well. But I am getting error with the unit test. Can someone explain the rational behind the input_dim = vocab_size vs vocab_size + 1

2

Which week;s assignment is this, because in W1 I am only finding input_dim=vocab_size…

3

Hi @mrinmoy_bhuyan

This is a good question and is related what is the “vocabulary”.

In essence it boils down to what is the value of the vocab_size in both cases.
In C3 W2 when mask=True, the 0 is reserved for mask, so your vocab size should increase by 1 (all previous words + mask) since mask value was previously not accounted for.

In C4 W1 :

There are a couple of important details to notice.

Padding has already been applied to the tensors and the value used for this is 0

So in this case, the vocabulary size does not increase by 1 (because it was accounted for already).

Cheers

2 Likes