Attention in the assgnment

Hi all,

In the assignment(W4), it look like we used Self-attention instead of LSH-attention.

I’ve replace tl.SelfAttention with tl.LSHSelfAttention but it didn’t work.

So, how to apply LSH attention?

Thank all!

Hi @20020069_Le_Thai_S_n

It’s not that easy. SelfAttention and LSHSelfAttention are different and you have to work around that (just changing the parameter would not work).