# Scaled\_dot\_product\_attention

**URL:** <https://community.deeplearning.ai/t/scaled-dot-product-attention/11732>\
**Category:** Sequence Models\
**Tags:** coursera-platform\
**Created:** [June 4, 2021, 10:31am UTC](https://community.deeplearning.ai/t/scaled-dot-product-attention/11732 "2021-06-04T10:31:59Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![xharles](https://avatars.discourse-cdn.com/v4/letter/x/b38774/32.png) [@xharles](https://community.deeplearning.ai/u/xharles)\
**Post date:** [June 4, 2021, 10:31am UTC](https://community.deeplearning.ai/t/scaled-dot-product-attention/11732/1 "2021-06-04T10:31:59Z")

</div>

According to the attention definition:  
 ![attention](https://global.discourse-cdn.com/dlai/original/2X/0/013832f79ef58e1e1ea5f1af4f2fc2ca4c0565fe.jpeg)

the dot product of `Q` and `K^{T}` should be scaled down by square root of `d_k`.  
However after got `d_k` from the `k.ndim`, I found that only divide `d_k` instead of its square root could pass the unit test. Could this be attribute to my implementation details, or the unit test itself?

---

<div class="post-metadata">

**Author:** ![martink91](https://avatars.discourse-cdn.com/v4/letter/m/85e7bf/32.png) [@martink91](https://community.deeplearning.ai/u/martink91)\
**Post date:** [June 4, 2021, 3:13pm UTC](https://community.deeplearning.ai/t/scaled-dot-product-attention/11732/2 "2021-06-04T15:13:31Z")

</div>

hi xharles,

I think .ndim is not the right property for this excercise as this gives you the dimension of the np array in terms of “how many axis does this tensor have”. → the value 2 → it’s a 2-dimensional array. But what you need is rather “how long is the dimension of that axis”, so .shape property might be helpful for you.  
I hope this hint helps you solving 🙂

P.s. the sqrt of the right solution is by coincidence equal to .ndim, but that only applies for the unittest example