In the lecture videos on self attention we had: [capture4] In the multi-head attention video we had: [capture5.PNG] Are the weights W^{<Q>}, W^{<K>}, W^{<V>} and W_i^{<Q>}, W_i^{<K>}, W_i^{<V>} different sets of weights?

Yes they are different sets of weights.

Clarification regarding attention and self attention

jonaslalin August 22, 2021, 1:37pm 5

Answered in