I have a general question about the class definition. For example, in the DecodeLayer class, in init function, self.mha1 is defined. In the call function in the class, when we use the self.mha1 function, how do we know which arguments the self.mha1 is supposed to take? I am confused about the definition of function/method in init and its use in call function.
Thanks.
Hello, in Tesnorflow multi-head attention link provided in the lab, it expalins the arguments that are used to call that function, here have a look:
Note here Q, K, V are the same ie. x.
I had seen the documentation previously. However, to clarify my original question, why do we define the self.mha1 and self.mha2 in the init function? It seems redundant to have the mha in both the init and call function?
In the innit definition they are initialising the mha, and then in the call is being called!
