W4 A1 number of heads vs

[Deepti_Prasad]

concatenation of head is to the weight, which if you see when head(i) is changed the attention weight is applied to all 3 multinhead attention, allowing the relative q, k and v value to be selected and getting the same output as the head(i) output.

Self-attention in the Transformer Network

We are not assigning q=k=v=x in encoder, but we create a matrix using the q, k and v relative to a given x, so it’s x position is determined from its ability to relate independently as well as positional to all other inputs(x’s), allowing x to be determined in decoder as well as transformer model using the similar matrix used in encoder for a given a input.