Hello,
I am receiving an error message that my output is not a tensor, but I don’t know why. The input to the final stack of encoders is a tensor (I’ve confirmed the type). Clearly I’m missing something - please can you help?
Additional note - since my previous message, I have tried the remaining exercises, none of them successfully, as the concepts are common to all. It is very disheartening to be so confused on the last assignment of the whole specialization, after so much work and effort. I’d greatly appreciate your help.
Many thanks,
Heidi
I suppose this exercise is the most challenging one in this specialization.
Let’s think about the structure of the Encoder at first.
We created an EncoderLayer int the Exercise 4. Transformer uses multiple-iteration of this layer as the Encoder. (You see N-times in the picture above.)
So, how to create multiple encoder layers in the Encoder ?
The trick is in the (_init_) in the Encoder class. As you see, self.enc_layers is created in here. This has a square bracket in front of “EncoderLayer” and “for loop (self.num_layers)”. This is a ListWrapper which includes multiple EncoderLayers as a list, and the number of layers is given by “self.num_layers”. An example is;
self.enc_layers =ListWrapper([<_main_.EncoderLayer object at 0x7fb308767370>, <_main_.EncoderLayer object at 0x7fb31bbba280>])
The above shows that there are 2 EncoderLayers in a list.
Then, the challenge is how to call EncoderLayers in a list ?
That’s the last layer of the Encoder. Basically, this is a “list” of EncoderLayers. So what we need to do is to specifying an index with a pair of square bracket, pass appropriate parameters, and iterate “self.num_layers” times.
Hope this helps.
Thank you, Nobu_Asai - that was very helpful. I have now been able to complete Ex 5 and 6. However, on Ex 7 I am receiving an error “Wrong values in outd”. I saw a similar thread had been posted on that error and it was due to a typo in the Decoder class, but I simply cannot find my mistake. Please may I send my code to you in a private message?
Thank you again,
Heidi
Due to the time zone issue, I will not have a chance to take a look tonight. As Ex7 is a short exercise, possible errors are, forgot sqrt(), error in slicing (2nd dimension is " :seq_len", not “seq_len”), and indexing to call decoding layer. I think 3rd one is not the case, as you fixed Ex5.
Of course, I can take a look tomorrow, but, other mentors may help you by then.
Hello - I don’t think it’s any of those typos, but thank you for the suggestions. Much appreciated.
Heidi
