1-Lipschitz continuous

Course Q&A Generative Adversarial Networks (GANS) Build Basic Generative Adversarial Networks
1

Hi All,

  1. I am unable to understand 1-Lipschitz continuous condition. Suppose if function id y=5x, then its derivative is 5 for any value of x, but even though its linear, its not growing fast, 1-L condition fails as derivative is 5. Also its norm will be 5 only.

  2. while implementing gradient penalty using regularization, interpolated image is used. But feature space for critic is so vast, I am assuming feature space as all kind of images real, fake. How taking only single interpolated image (for a real and fake image pair ) will ensure that function is 1-L continuous. I mean, how concept of checking 1-L continuity using all feature space point , can be reduced to just using one interpolated image.

2

Maybe a gradient of 5 seems “not too fast” to you, but the point of 1-L continuity is that it defines what “not growing too fast” means as the gradients having 2-norm <= 1. With that definition a gradient of 5 is growing way too fast, so that function is not 1-L continuous.

That is just showing how to do the gradient penalty W loss computation for one input. Then we apply that to the entire batch of inputs. You’ll see this in the assignment.