# C5 W4 Lab1 E3 **URL:** \ **Category:** Sequence Models\ **Tags:** week-module-4, coursera-platform\ **Created:** [March 24, 2025, 3:30pm UTC](https://community.deeplearning.ai/t/c5-w4-lab1-e3/790405 "2025-03-24T15:30:03Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![ejwalter](https://avatars.discourse-cdn.com/v4/letter/e/f6c823/32.png) [@ejwalter](https://community.deeplearning.ai/u/ejwalter)\ **Post date:** [March 24, 2025, 3:30pm UTC](https://community.deeplearning.ai/t/c5-w4-lab1-e3/790405/1 "2025-03-24T15:30:03Z")
I am getting a wrong shape error for my scaled\_dot\_product\_attention function: —\> 59 assert tuple(tf.shape(weights).numpy()) == (q.shape[0], k.shape[1]), f"Wrong shape. We expected ({q.shape[0]}, {k.shape[1]})" 60 assert np.allclose(weights, [[0.2589478, 0.42693272, 0.15705977, 0.15705977], 61 [0.2772748, 0.2772748, 0.2772748, 0.16817567], AssertionError: Wrong shape. We expected (3, 4) Since I can add the mask to scaled\_attention\_logits, I don’t think that is the wrong shape. I did transpose k for the first matrix multiplication. I did not transpose v in the second matmult. It doesn’t actually say where the problem is occurring in the function. Any ideas what I could be doing wrong? ---
**Author:** ![paulinpaloalto](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/paulinpaloalto/32/187_2.png) [@paulinpaloalto](https://community.deeplearning.ai/u/paulinpaloalto)\ **Post date:** [March 24, 2025, 4:10pm UTC](https://community.deeplearning.ai/t/c5-w4-lab1-e3/790405/2 "2025-03-24T16:10:16Z")
Notice that it’s “throwing” about the shape of the `weights`, not the `output`. I added print statements to show all the relevant shapes in this function and here’s what I see when I run that test cell: ```auto q.shape (3, 4) k.shape (4, 4) v.shape (4, 2) matmul_qk.shape (3, 4) matmul_qk = [[2. 3. 1. 1.] [2. 2. 2. 1.] [2. 2. 0. 1.]] dk 4.0 type(scaled_attention_logits) scaled_attention_logits.shape (3, 4) attention_weights.shape (3, 4) attention_weights = [[0.2589478 0.42693272 0.15705977 0.15705977] [0.2772748 0.2772748 0.2772748 0.16817567] [0.33620113 0.33620113 0.12368149 0.2039163]] sum(attention_weights(axis = -1)) = [[1.0000001] [1.] [1.]] output.shape (3, 2) output = [[0.74105227 0.15705977] [0.7227253 0.16817567] [0.6637989 0.2039163]] q.shape (3, 4) k.shape (4, 4) v.shape (4, 2) matmul_qk.shape (3, 4) matmul_qk = [[2. 3. 1. 1.] [2. 2. 2. 1.] [2. 2. 0. 1.]] dk 4.0 type(scaled_attention_logits) scaled_attention_logits.shape (3, 4) mask.shape (1, 3, 4) applying mask = [[[1 1 0 1] [1 1 0 1] [1 1 0 1]]] attention_weights.shape (1, 3, 4) attention_weights = [[[0.3071959 0.5064804 0. 0.18632373] [0.38365173 0.38365173 0. 0.23269653] [0.38365173 0.38365173 0. 0.23269653]]] sum(attention_weights(axis = -1)) = [[[1.] [1.] [1.]]] output.shape (1, 3, 2) output = [[[0.6928041 0.18632373] [0.61634827 0.23269653] [0.61634827 0.23269653]]] All tests passed ``` One way to debug this would be to add equivalent statements for at least the weights in your code and compare your results to what I show above. That should shed some light for where to look in more detail. ---
**Author:** ![ejwalter](https://avatars.discourse-cdn.com/v4/letter/e/f6c823/32.png) [@ejwalter](https://community.deeplearning.ai/u/ejwalter)\ **Post date:** [March 24, 2025, 5:20pm UTC](https://community.deeplearning.ai/t/c5-w4-lab1-e3/790405/3 "2025-03-24T17:20:45Z")
Thanks. I added in some print statements and the problem seems to happen when I add on the mask. And when I try to print the shape of the mask it says ‘NoneType’ object has no attribute ‘shape’. The mask is an input variable, though, so shouldn’t it have the correct shape when it’s passed in? ---
**Author:** ![ejwalter](https://avatars.discourse-cdn.com/v4/letter/e/f6c823/32.png) [@ejwalter](https://community.deeplearning.ai/u/ejwalter)\ **Post date:** [March 24, 2025, 5:49pm UTC](https://community.deeplearning.ai/t/c5-w4-lab1-e3/790405/4 "2025-03-24T17:49:46Z")
My attention weights have the same values as yours but the wrong shape. attention\_weights: tf.Tensor( [[[0.2589478 0.42693272 0.15705977 0.15705977] [0.2772748 0.2772748 0.2772748 0.16817565] [0.33620113 0.33620113 0.12368149 0.2039163]] [[0.2589478 0.42693272 0.15705977 0.15705977] [0.2772748 0.2772748 0.2772748 0.16817565] [0.33620113 0.33620113 0.12368149 0.2039163]] [[0.3071959 0.5064804 0. 0.18632373] [0.38365173 0.38365173 0. 0.23269653] [0.38365173 0.38365173 0. 0.23269653]]], shape=(3, 3, 4), dtype=float32) ---
**Author:** ![paulinpaloalto](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/paulinpaloalto/32/187_2.png) [@paulinpaloalto](https://community.deeplearning.ai/u/paulinpaloalto)\ **Post date:** [March 24, 2025, 7:12pm UTC](https://community.deeplearning.ai/t/c5-w4-lab1-e3/790405/5 "2025-03-24T19:12:00Z")
Yes, but note that the mask is not always included, right? There are two test cases: one w mask and one w/o. ---
**Author:** ![paulinpaloalto](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/paulinpaloalto/32/187_2.png) [@paulinpaloalto](https://community.deeplearning.ai/u/paulinpaloalto)\ **Post date:** [March 24, 2025, 7:14pm UTC](https://community.deeplearning.ai/t/c5-w4-lab1-e3/790405/6 "2025-03-24T19:14:22Z")
In the w mask case, I get shape 1,3,4, but it looks like you get shape 3,3,4. So how could that happen? Sorry I am away from my computer for the next couple of hrs, so cannot look at the code to give a better hint …. ---
**Author:** ![ejwalter](https://avatars.discourse-cdn.com/v4/letter/e/f6c823/32.png) [@ejwalter](https://community.deeplearning.ai/u/ejwalter)\ **Post date:** [March 24, 2025, 8:58pm UTC](https://community.deeplearning.ai/t/c5-w4-lab1-e3/790405/7 "2025-03-24T20:58:12Z")
It seems like the padding mask adds an extra dimension and that is what is throwing things off. I tried using tf.squeeze to get rid of the extra dimension, but that didn’t work - it then said my weight values were incorrect. Which doesn’t really surprise me because it is obviously deliberately added. But I definitely only need to use the values from two out of the three dimensions. ---
**Author:** ![ejwalter](https://avatars.discourse-cdn.com/v4/letter/e/f6c823/32.png) [@ejwalter](https://community.deeplearning.ai/u/ejwalter)\ **Post date:** [March 24, 2025, 9:08pm UTC](https://community.deeplearning.ai/t/c5-w4-lab1-e3/790405/8 "2025-03-24T21:08:37Z")
Okay, I figured it out. I was adding the mask twice - once where I was supposed to and again inside the softmax (I was calling create\_padding\_mask and making a new mask inside the softmax instead of just using the mask provided to the function). ---
**Author:** ![paulinpaloalto](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/paulinpaloalto/32/187_2.png) [@paulinpaloalto](https://community.deeplearning.ai/u/paulinpaloalto)\ **Post date:** [March 25, 2025, 12:47am UTC](https://community.deeplearning.ai/t/c5-w4-lab1-e3/790405/9 "2025-03-25T00:47:36Z")
It’s good news that you found the issue! Thanks for confirming.