C1_W3_ProteinGAN_R1_regularization

Course Q&A Generative Adversarial Networks (GANS) Build Basic Generative Adversarial Networks
1

As explained in the beginning of the lab:

Adapting the implementation of non-saturating loss with R1 regularization greatly improves the performance of the GAN as demonstrated in the figure below.

I noticed 2 points which could potentially help in providing better description of the lab for later use:

  • In the lab, I could not find any cell referring to and/or calculating R1 regularization for the given dataset and pre-trained model.
    Where can we find more information regarding the regularization part in this lab?

  • R1 regularization was originally proposed in Kevin Roth paper and the authors of the aforementioned paper, referenced it to the main authors:

bild
bild720×782 135 KB

Cheers,

2

Thanks, @mrgransky

One comment on your first point about the R1 regularization not being included in the lab: The way I’m reading it, esp. based on this sentence, “Despite the implementation of spectral normalization and WGAN loss, the optimization of ProteinGAN did not lead to convergence”, I think the authors of the lab are basically saying, “This lab didn’t end up converging as nicely as we would have liked”, but then they go on to say that based on the papers they cited, it should work better to use R1 regularization, but they are leaving it “as an exercise for the reader” to try R1 regularization if they want to experiment.