Hi, Pavel.
It looks like you are already a sophisticated and experienced user of PyTorch when you came to this course. If you’ve looked around at the Deep Learning courses, you’ve probably noticed that all of them except GANs use TensorFlow. So this course is set up to assume that the student is seeing PyTorch for the very first time, as you can tell from the torch tutorial in Week 1. So perhaps they didn’t want to get into the level of detail at which you’ve explained things here and are trying to keep things relatively simple.
I’m pretty sure from your explanation that you already understand the fundamental asymmetry here in that the gradients of the generator depend on those of the discriminator, but you can train the discriminator without the generator’s gradients. Here’s another thread that discusses that point, but I’m guessing it’s already clear to you.
Here’s a previous thread which discusses the difference between no_grad and “detach”, but there again your points are already at a more sophisticated level than the discussion there.
Also note that this is a little awkward because you are showing the solution code in your post. Normally I would delete the source code and leave a marker about it, since we are not supposed to share solution code in a public way (spoils the fun for everyone else and leads to cheating). But without the code, then your points are a lot harder for people to understand. So what I did instead is to “Unlist” this thread, which means that no-one but you and the mentors can see it. That also sort of defeats the purpose. Sorry, just trying to work within the intent of the rules here …
The other thing I could try would be to forward this thread to the course developers, but I’m not sure they are still listening here.
Regards,
Paul