# Dec\_padding\_mask

**URL:** <https://community.deeplearning.ai/t/dec-padding-mask/564273>\
**Category:** NLP with Attention Models\
**Tags:** week-module-2\
**Created:** [February 3, 2024, 6:29am UTC](https://community.deeplearning.ai/t/dec-padding-mask/564273 "2024-02-03T06:29:42Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![blackdragon](https://avatars.discourse-cdn.com/v4/letter/b/9dc877/32.png) [@blackdragon](https://community.deeplearning.ai/u/blackdragon)\
**Post date:** [February 3, 2024, 6:29am UTC](https://community.deeplearning.ai/t/dec-padding-mask/564273/1 "2024-02-03T06:29:42Z")

</div>

For exercise 5 (next\_word function), why can’t we pass in the output into the create\_padding\_mask function when defining dec\_padding\_mask? Isn’t “output” here the decoder input which is what is supposed to be sent?

---

<div class="post-metadata">

**Author:** ![arvyzukai](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/arvyzukai/32/165165_2.png) [@arvyzukai](https://community.deeplearning.ai/u/arvyzukai)\
**Post date:** [February 6, 2024, 6:01pm UTC](https://community.deeplearning.ai/t/dec-padding-mask/564273/2 "2024-02-06T18:01:31Z")

</div>

Hi @blackdragon

The name `dec_padding_mask` might have mislead you. The `dec_padding_mask` is used in the second Multi-Head attention (Cross-Attention):

 ![image](https://global.discourse-cdn.com/dlai/original/3X/9/2/923452423c9fe3a16f57735d0a2596848f7dbc30.png)

It is used to let the decoder know which encoder inputs where padding.  
For example, if the English sentence is “I love learning \<pad\> \<pad\> \<pad\> \<pad\> \<pad\> \<pad\>” the Encoder encodes this sequence with 8 tokens long (1, 8, n\_units) (batch\_size, seq\_length, feature\_size).  
So, this mask informs the decoder that tokens starting with 4 are padding tokens and it should not pay any attention to them when generating the German sentence.

Cheers