Hi;
İn assignment week 4 of C5(Sequence Model) at Transformer Network
I would like understand logic behind method self.enc_layers definition.
self.enc_layers = [EncoderLayer(embedding_dim=self.embedding_dim,
num_heads=num_heads,
fully_connected_dim=fully_connected_dim,
dropout_rate=dropout_rate,
layernorm_eps=layernorm_eps)
for _ in range(self.num_layers)]
My python knowledge may not enough to understand this structure, Can someone explain in details.
Thanks a lots.
Hello @Erkan_Yucekaya! I hope you are doing well.
Thank you for your post. However, you posted this in the wrong category. Currently, it is in the General category. Kindly change the category to the relevant course (DLS 5 in this case) and our mentors will be happy to assist you.
Please note that posting in the wrong category may decrease the chances of your post being noticed by our mentors and you may not receive a timely response. Therefore, it’s important to ensure that your post is in the correct category.
You can change the category by clicking the pencil icon on the right side of the title, as shown in the below image.
Best,
Saif.
That code is in the constructor for the Encoder() class.
enc_layers is a list of EncoderLayer() objects. They’re created by the for-loop that is embedded within the square brackets.
You implemented the EncoderLayer() class earlier in the notebook.
The EncoderLayer() class uses the Keras MultiHeadAttention() class. You can read about that from the link in the notebook instructions at the start of Section 4.
1 Like