Odd lack of correlation between my discriminator and the grader's discriminator

Course Q&A TensorFlow: Advanced Techniques Specialization Generative Deep Learning with TensorFlow
1

Before submitting images to the grader for the GAN Hand assignment, I first saw what my own discriminator thought. I got the following result:

<tf.Tensor: shape=(16, 1), dtype=float32, numpy=
array([
[7.9994911e-01],
[9.9239588e-01],
[9.4112718e-01],
[7.7543765e-01],
[9.4984710e-01],
[7.8973532e-01],
[3.0495387e-01],
[3.8384840e-01],
[5.7092643e-01],
[3.2548344e-01],
[5.2061573e-02],
[1.8261197e-01],
[1.8240607e-04],
[4.3043430e-05],
[9.8573253e-02],
[7.2770426e-04]], dtype=float32)>

I was not anticipating a pass, since I had only 6 images above the required threshold a few more that were at least within a factor of 2 and then 4 I just randomly placed in at the end to get 16 images. However, mirabile dictu, I passed with these results:

score for image hand0.png: 0.894
score for image hand1.png: 0.595
score for image hand2.png: 0.797
score for image hand3.png: 0.713
score for image hand4.png: 0.375
score for image hand5.png: 0.729
score for image hand6.png: 0.325
score for image hand7.png: 0.428
score for image hand8.png: 0.813
score for image hand9.png: 0.739
score for image hand10.png: 0.302
score for image hand11.png: 0.382
score for image hand12.png: 0.746
score for image hand13.png: 0.506
score for image hand14.png: 0.317
score for image hand15.png: 0.753

Where of my “solid 6”, only 4 passed. Luckily, 2 from my ‘closish’ group passed. But what was truly surprising was that two images that my discriminator rejected with greater than 99% accuracy, were accepted by the grader.

I suspect this shows the sensitivity of the discriminator to the exact series of noisily generated images it trains with. I also suspect, that the best way to ‘pass’ is to take advantage of the ability to submit as many times as one likes. Just submit all 31 images (or more if you save ‘good’ images during training), see if there is some set of 8 images that the grader has >60% confidence in, and make a final submission including those images.

The behavior of the grader is correlated with what looks good to eye or one’s discriminator, but IMHO not so strongly correlated that one can rely upon either one.

1 Like
2

Hi Steven,

Who knows how the grader is trained ;-). In the assignment we are training the discriminator jointly with the generator, maybe our discriminator needs more epochs to identify the fake images correctly.

I remember that I choosed the ones i like more to pass, and I identified a pattern that some images get better results than others :-).

Congrats to pass the assignment!!! And to finish the course. For me it was the best TensorFlow course I’ve ever did.

If you are interested in GAN’s I wrote a serie of articles at:
https://medium.com/@peremartra/list/gans-from-zero-to-hero-d8e6cb773f93

I tried to go a little bit beyond the course, building a conditional GAN in the last article, and explaining how to use TPUs to accelerate the process of training the GAN.

Pere.

3

Hi Pere,

Thanks, I will give those articles a read. I think the real lesson here is why guess what the grader will do? We have unlimited submissions, so why not just submit everything and then decide which images the grader likes best?

  • Steven
4

hey can I know did you choose grader image based on the images saved in the file section or you choose by what you felt look good as a hand??

5

I had an initial ‘dummy’ submission where I submitted ALL my images to the grader Then (as I recall) I picked the 8(?) best and resubmitted a set containing only them.

1 Like
6

Nice strategy! Better than the one I followed for sure!

And the selected hands had something in common?

7

Steven we are suppose to select 16 images and can I know how many epochs you trained your model to run? was your model training too slow?

I wanted to confess some of the question you have posted in the community is the very doubts I have had, but I didn’t see many of them not being replied by anyone. I am waiting to see what other leaders and mentors have to say on those questions.

Thank you
DP

8

Deepti - Sorry, I don’t recall how many epochs I trained and can’t find my old code (argh!), but I’m pretty sure I just used the suggested number of epochs. Possibly, I tried 2x as many, but definitely not more. Looking back, you’re correct, we did submit 16 images (but maybe only 8 needed to pass the quality threshold?? Sorry, I really don’t recall). However, the point of my suggestion (and this I do recall) is that I was not able to accurately determine which images the grading algo would like. Yes, my guesses were better than random, but still not entirely accurate. So, I took advantage of the facts that 1. We have unlimited submissions 2. I had generated more than 16 final images. So, I used multiple submissions to obtain scores for all the images I had generated, and then resubmitted the a set of the ‘best’ images as a final submission. Hope this helps. Sorry I can’t say how many training epochs I used for certain, but I don’t believe it was something I tweaked too hard…

1 Like
9

So basically you submitted all the images and then checked which one at better accuracy at matching close to the hands and then selected the hand for second submission???

10

Yup… That’s exactly what I did. My guesses as to which images the grader would like best weren’t always right. But, since we have unlimited submissions, I didn’t need to rely on them.

1 Like
11

actually that was a brilliant idea, Thank you for all the response everyone here gives in learning and growing knowledge. Please ping me if your post questions get answered or replied as I have the same doubts regarding GANs and even neural style transfer