C5W4 Transformers Encoder Decoder doubt

In the encoder and decoder code I see 3 self.layernorm s which essentially are defined to be the same thing. Is there any reason why can’t we have 1 variable self.layernorm and use it 3 times in the call() code?

self.layernorm1 = LayerNormalization(epsilon=layernorm_eps)
self.layernorm2 = LayerNormalization(epsilon=layernorm_eps)
self.layernorm3 = LayerNormalization(epsilon=layernorm_eps)

That’s because they are three different instances, one for each layer. They’re not functions, they’re separate objects.

1 Like