# Parallelism At Decoder Layer In Transformers **URL:** \ **Category:** Sequence Models\ **Tags:** coursera-platform\ **Created:** [June 21, 2023, 7:55am UTC](https://community.deeplearning.ai/t/parallelism-at-decoder-layer-in-transformers/360422 "2023-06-21T07:55:20Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Kutay\_Cavdar](https://avatars.discourse-cdn.com/v4/letter/k/7feea3/32.png) [@Kutay\_Cavdar](https://community.deeplearning.ai/u/Kutay_Cavdar)\ **Post date:** [June 21, 2023, 7:55am UTC](https://community.deeplearning.ai/t/parallelism-at-decoder-layer-in-transformers/360422/1 "2023-06-21T07:55:20Z")
I have understood how parallelism in the encoder layer works. But there is one thing I can’t wrap my head around and that is why we don’t do all computations in parallel in the decoder layer. I remember Andrew said, “Since we know what the true y output is, we can present each node in the decoder with correct previous words and see how well it can guess what the next word is”. In that case, we could use look\_ahead\_mask in each decoder node and can present the true output multiple times to calculate the loss function and do backpropagation instead of waiting for each node to make a prediction. Thank you for your time. ---
**Author:** ![balaji.ambresh](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/balaji.ambresh/32/128697_2.png) [@balaji.ambresh](https://community.deeplearning.ai/u/balaji.ambresh)\ **Post date:** [June 21, 2023, 10:25am UTC](https://community.deeplearning.ai/t/parallelism-at-decoder-layer-in-transformers/360422/2 "2023-06-21T10:25:12Z")
Just like an encoder, a decoder is also made of multiple layers. Please explain what you mean by this: > [@Kutay\_Cavdar](#): > > instead of waiting for each node to make a prediction. ---
**Author:** ![Kutay\_Cavdar](https://avatars.discourse-cdn.com/v4/letter/k/7feea3/32.png) [@Kutay\_Cavdar](https://community.deeplearning.ai/u/Kutay_Cavdar)\ **Post date:** [June 21, 2023, 2:25pm UTC](https://community.deeplearning.ai/t/parallelism-at-decoder-layer-in-transformers/360422/3 "2023-06-21T14:25:32Z")
In the encoder, we don’t wait for the third word in the sentence in order to create an encoding for the fourth word. However, in the decoder layer, we need the prediction from the previous nodes to be able to make a prediction at some specific node. If we already have the correct translation at training, why not use look ahead mask and correct words from our data to be able to calculate loss functon & gradients faster? ---
**Author:** ![balaji.ambresh](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/balaji.ambresh/32/128697_2.png) [@balaji.ambresh](https://community.deeplearning.ai/u/balaji.ambresh)\ **Post date:** [June 21, 2023, 5:33pm UTC](https://community.deeplearning.ai/t/parallelism-at-decoder-layer-in-transformers/360422/4 "2023-06-21T17:33:33Z")
During training, decoder attention happens in parallel since (as you pointed out), we have the correct translation. Consider the task of translating english to spanish. Say we have already predicted the first 3 words. To predict the 4th word, we need the complete information about the english sentence and information about the first 3 spanish words. This is what the look ahead mask is meant to help with. Please try the programming assignment and reply to this thread if something is unclear. ---
**Author:** ![Kutay\_Cavdar](https://avatars.discourse-cdn.com/v4/letter/k/7feea3/32.png) [@Kutay\_Cavdar](https://community.deeplearning.ai/u/Kutay_Cavdar)\ **Post date:** [June 23, 2023, 6:21pm UTC](https://community.deeplearning.ai/t/parallelism-at-decoder-layer-in-transformers/360422/5 "2023-06-23T18:21:59Z")
Oh okay so we form the words in the translation simultaneously in training right? ---
**Author:** ![balaji.ambresh](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/balaji.ambresh/32/128697_2.png) [@balaji.ambresh](https://community.deeplearning.ai/u/balaji.ambresh)\ **Post date:** [June 23, 2023, 7:21pm UTC](https://community.deeplearning.ai/t/parallelism-at-decoder-layer-in-transformers/360422/6 "2023-06-23T19:21:18Z")
Your understanding is correct: > [@balaji.ambresh](#): > > During training, decoder attention happens in parallel since (as you pointed out), we have the correct translation. ---
**Author:** ![Kutay\_Cavdar](https://avatars.discourse-cdn.com/v4/letter/k/7feea3/32.png) [@Kutay\_Cavdar](https://community.deeplearning.ai/u/Kutay_Cavdar)\ **Post date:** [June 24, 2023, 4:37pm UTC](https://community.deeplearning.ai/t/parallelism-at-decoder-layer-in-transformers/360422/7 "2023-06-24T16:37:59Z")
Thank you for your time. Have a good day.