In a preceding lecture, the instructor said that while training the generator, the cost function will always be given true label 1(real) because we want fakes by the generator to be as real as possible (close to 1). So if the Discriminator outputs 0(fake) then we penalize the cost function with a large value. Later in this lecture, the instructor says that if Discriminator is superior to the Generator, then it will always output fake for every image given by the Generator. My question is, now, if the Discriminator always gives out fake(0) as output, and the true label to the cost function is always 1(real), it will penalize the cost with a large value. And in the other case, if it says real(1) then the cost is not penalized. In the first case we have a large cost and in the latter we almost 0 cost. Doesn’t low cost signify that the generator is doing well, and a high cost signifies that the generator still has to improve a lot? Isn’t a large cost now useful for improving the Generator?
@Gaurav_Niranjan, you’re right that if the Discriminator is very good so that it can almost always return fake for every image from the Generator, then the generator’s cost will be very large - so it knows it needs to improve. But, the problem is it’s hard for it to get better because no matter what it tries, it gets (roughly) the same large cost and it’s hard for it to tell what it needs to do to get better.