DotProductAttention UNC_C1: Is mask boolean or numeric?

Course Q&A Natural Language Processing NLP with Attention Models
1

Hi there,

I am confused whether mask is supposed to be boolean or numeric –

The in notebook example, as well as code in rest the assignment (particularly UNQ_C3) implies it should be boolean, but the unit test code for UNQ_C1 supplies a numeric mask:

image

Please let me know if I have caught a bug, or whether I am misunderstanding the instructions.

Thank you :slight_smile:

2

Followup:

I am now wondering if the unit tests for UNC_C1 are perhaps outdated.

My function perfectly matches the output in the notebook:

image
image705×181 19 KB

And I now notice that the expected outputs in the unit tests are a bit too perfect (see image in original post)…

I thought attention was supposed to be a weighted sum of the key/value embeddings that “match” the query embeddings.

From this understanding one would expect the output numbers to be less perfect than “1.0”, “2.0”, etc.

Am I missing something here? I would love some clarity, thank you!

3

FINAL UPDATE:

I ended up passing the assignment with a 90% score!

Turns out ignoring the failing unit test for UNC_C1 had no downstream effects, which leads me to believe the unit tests for that task might be a bit off.

It would be great for the course staff to look into this issue.

Thanks again!

1 Like