Course 5 Week 4 A1 transformer subclass v1

Course Q&A Deep Learning Specialization Sequence Models
1

in the call function of exercise 5
for i in range(self.num_layers):
x = self.enc_layers[i](x, training, mask)
wouldn’t that override the value of x every single time? and eventually only the output of the last encoder is stored in x

2

Yes.

3

thank you for your response :slight_smile: is that logically correct? (if yes then what is the point of the loop) and if not, what should be done instead?

4

That’s the way neural networks work, right? You have a sequence of layers and the input to the next layer is the output of the previous layer. And what you care about is the final output. Granted it’s typically not something you can do with a “for” loop, but this is a special case.

1 Like
5

BTW you filed this under DLS Course 4 ConvNets, but I don’t remember anything like that in ConvNets Week 4. Did you perhaps mean DLS C5 Sequence Models?

I took a look at that notebook to refresh my memory and they specifically talk about having a “stack” of encoder layers in that model. So this is a bit of special case, but they do intend to pass the input through a serial chain of encoder layers. It’s right there in the description. So given that you have an array of the layers, using a “for” loop is a compact way to express what they want to achieve there.

1 Like
6

thank you for the response!, sorry for the confusion, i did indeed mean course 5 week 4 A1 which is about transformers! it has been great help !

1 Like