Course 5 Week 4 A1 transformer subclass

Course Q&A Generative AI with Large Language Models
1

Maybe this is a simple question, but why do we need to define the encoder layers like this

  self.enc_layers = [EncoderLayer(embedding_dim=self.embedding_dim,
                                          num_heads=num_heads,
                                          fully_connected_dim=fully_connected_dim,
                                          dropout_rate=dropout_rate,
                                          layernorm_eps=layernorm_eps) 
                            for _ in range(self.num_layers)]

and then call it by this

       for i in range(self.num_layers):

                      x = self.enc_layers[i](x,training, mask)

Can we define it like this:

                  self.enc_layer = EncoderLayer(embedding_dim=self.embedding_dim,
                                                  num_heads=num_heads,
                                                  fully_connected_dim=fully_connected_dim,
                                                  dropout_rate=dropout_rate,
                                                  layernorm_eps=layernorm_eps)

and then call it like this

        for i in range(self.num_layer):

              x = self.enc_layer(x,training, mask)
2

In this loop:

for i in range(self.num_layers):

          x = self.enc_layer(x,training, mask)

where are you using i?

The purpose of i is to loop through all encoder layers and you are not doing it in your loop!

3

Hello, I mean the i is just for dummy loop for execute x = self.enc_layer(x,training, mask) for n=self.num_layer times, so this should have the same result as stack the same number of EncoderLayer in a list and the send x in this list and go through all the layers inside

4

From this I understand that that there are i layers to be run here, indexed by i!