does anyone else agree the content of week 3 lacks the quality and level of detail seen so far in the course?
for instance, while presenting the bert architecture diagram there is no explanation of the nomenclature of what the outputs referred as T1, T2 mean.
another example, while explaining the T5 architecture, 3 blocks of layers are presented and as they are explained they are referred as “in the left we have…”, “in the middle we have”, “to the right we have…” but how these blocks play together?
The video Multi task training strategy is even chopped at the end.
1 Like
Hi gerardo,
Thanks for your comment. I also feel that the course could use an update with clearer explanations, particularly for those who are not already familiar with the transformer architectures that are discussed, and also because developments in the field of transformer architectures are going very fast.
As for the presentation of the BERT architecture, it would have been good if Younes had mentioned explicitly that T1, T2, etc. are the final hidden vectors of the model, although this to some extent becomes clear at around 3:11 in the video and in the BERT Objective video. It would also have been good had a visual depiction been provided of the way in which the Ti vectors lead to predictions. According to the BERT paper, the Ti vectors are fed directly into an output softmax over the vocabulary without the dense layer and embedding layer multiplication mentioned by Younes.
As for the T5 architecture, it could have been explained more clearly that for the original T5 paper three different model structures were tested - the three architectures that are shown and described. The paper concludes that the Encoder-Decoder structure (the structure on the left) worked best - as is implicitly stated at around 3:08 in the video.
I also agree that the cut-off of the explanation in the multi-task training strategy video is rather abrupt, even if it does seem to have reached its end.
So yes, it would be good if the course were updated. In any case, thanks a lot for your valuable feedback! I’ll make a note of this to the team.
I agree. The lectures’ texts seem less fluid and of lower quality, full of filler words and shallow explanations. It is a different experience from previous courses/weeks. I don’t mean to just criticize. I hope the feedback helps to improve the courses’ quality.
@gerardo I agree! The quality of course 4 seems to degrade week over week, and overall isn’t on par with the other 3 courses in this specialization.