# Why is extra dim needed in create\_padding\_mask in Transformer Network assignment **URL:** \ **Category:** Sequence Models\ **Tags:** coursera-platform\ **Created:** [July 21, 2021, 2:37am UTC](https://community.deeplearning.ai/t/why-is-extra-dim-needed-in-create-padding-mask-in-transformer-network-assignment/22340 "2021-07-21T02:37:36Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![kechan](https://avatars.discourse-cdn.com/v4/letter/k/aca169/32.png) [@kechan](https://community.deeplearning.ai/u/kechan)\ **Post date:** [July 21, 2021, 2:37am UTC](https://community.deeplearning.ai/t/why-is-extra-dim-needed-in-create-padding-mask-in-transformer-network-assignment/22340/1 "2021-07-21T02:37:36Z")
In the programming exercise: I was thinking the mask should have the same shape as the input “seq”. Why are 2 new dimensions are needed? Also, as a result, the 2 different softmax calculated next have different shapes? [3, 5] vs. [3, 1, 3, 5]?? ---
**Author:** ![edwardyu](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/edwardyu/32/82_2.png) [@edwardyu](https://community.deeplearning.ai/u/edwardyu)\ **Post date:** [July 23, 2021, 8:53am UTC](https://community.deeplearning.ai/t/why-is-extra-dim-needed-in-create-padding-mask-in-transformer-network-assignment/22340/2 "2021-07-23T08:53:35Z")
Mask is used in _scaled\_dot\_product\_attention_, and added to (q multiplied by k transpose), as a result, its shape has to be compatible with (batch\_size, header\_nums, seq\_len, seq\_len). By the way, the softmax calculation example w/ and w/o mask is just to demonstrate how it works. In fact, we won’t add mask to input sequence x directly in MultiHeadAttention. ---
**Author:** ![paulinpaloalto](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/paulinpaloalto/32/187_2.png) [@paulinpaloalto](https://community.deeplearning.ai/u/paulinpaloalto)\ **Post date:** [March 15, 2025, 7:06pm UTC](https://community.deeplearning.ai/t/why-is-extra-dim-needed-in-create-padding-mask-in-transformer-network-assignment/22340/3 "2025-03-15T19:06:26Z")