WGAN Weight Initialization

Hi everyone! I was wondering if there was anything against initializing the weights of the Generator and Discriminator/Critic differently. What I mean is, let’s say using Pytorch, using kaiming_normal_() for the Discriminator but then just normal_() for the Generator. Or other combos. Usually I see them initialized the same way. Thank you!

1 Like

It’s an interesting question. The general rule for ML/DL is that there is no single initialization strategy which works best in all cases. I took the GANs specialization 3 or 4 years ago, but don’t remember the initialization algorithms used. I know that the course creators for the GANs specialization are experts in the field, so they probably made those choices based on their experience and knowledge of the state of the art in the field. But it’s an interesting point to question why the choices were made the way they were. You can try some experiments and see if you can notice any improvement or perhaps degradation when trying different combinations. If you find anything either way, that would be interesting knowledge to share. Science! :nerd_face: