GANs Course 2, Week 1, C2W1_Assignment

This sounds like a resource limitation problem on the AWS VMs that they use to run the notebooks. When you run the training for a lot of iterations, the memory footprint can get very large which slows things down and can interfere with submitting to the grader. One of the palliatives mentioned on the FAQ Thread for the “no submit button” case is to do “Kernel → Restart and Clear Output” to discard all the generated results and memory footprint. Of course you also lose the results of the training when you do that. Sigh … Unless you have your own captive hardware for running training, this is a problem with pretty much any online environment. If you are doing this kind of thing seriously, you need to add logic to checkpoint (save) the trained weights periodically and then structure the training logic so that you can restart and continue from a previously trained set of weights. That is beyond the scope of the course materials, but it’s easy to “google up” some instructions to get started down that path.

1 Like