I may not be understanding transformers at a conceptual level, but I don’t think the lecture covers my question. I noticed that, when we implemented the encoder, we didn’t keep track nor update the attention weights. However, we did that in the decoder. It seems that the weights weren’t used later …

W4 - Assignment: Why do we only update the attention weights in the decoder, but not in the encoder?

Course Q&A Deep Learning Specialization Sequence Models

balaji.ambresh December 2, 2022, 12:35pm 2

Please see this topic. It’s good not to create duplicate topics. You can edit a reply and update your questions. Here’s community user guide to get started.

Topic		Replies	Views
Conceptual Questions about Transformers Sequence Models	13	654	April 23, 2023
Course 5 Week 4 Assignment: Why are attention weights returned in DecoderLayer Sequence Models	1	730	June 29, 2022
Week 4: Transformer Network (test time intuition) Sequence Models	1	514	April 21, 2022
Transformer Architecture Assignment Sequence Models	4	561	May 11, 2022
General Understanding of Transformer Encoder and Decoder blocks NLP with Attention Models week-3	7	734	January 22, 2024

W4 - Assignment: Why do we only update the attention weights in the decoder, but not in the encoder?

Related topics