Attention in the assgnment

Course Q&A Natural Language Processing NLP with Attention Models
1

Hi all,

In the assignment(W4), it look like we used Self-attention instead of LSH-attention.

image
image894×27 3.05 KB

I’ve replace tl.SelfAttention with tl.LSHSelfAttention but it didn’t work.

image
image1408×607 33.8 KB

So, how to apply LSH attention?

Thank all!

2

Hi @20020069_Le_Thai_S_n

It’s not that easy. SelfAttention and LSHSelfAttention are different and you have to work around that (just changing the parameter would not work).

Cheers