C1W1 Training Loop

Dhia_Znaidi · May 21, 2023, 10:39pm

Hi,
I have a question regarding the usefuleness of using the retain_graph = True when doing the backward step using the discriminator.
Is it because because the discriminator will be used later on when training the generator ?
Thanks.

Nithin_Skantha_M · May 22, 2023, 12:26am

Hi Dhia_Znaidi!
Hope you are doing well.

Yes, the retain_graph=True argument is used during the backward pass of the discriminator to retain the computational graph. This is done because the same discriminator network will be used again when training the generator.

By retaining the graph, you ensure that the gradients can be computed and propagated through the discriminator during the subsequent backward pass when training the generator. Without retain_graph=True, the computational graph would be discarded after the first backward pass (the usual scenario, due to memory efficiency scenarios).

Regards,
Nithin

Dhia_Znaidi · May 22, 2023, 9:00am

Great Thanks for the clarification

Topic		Replies	Views
retain_graph=True? Build Basic Generative Adversarial Networks week-1	3	569	June 1, 2022
retain_graph=True and disc_opt.zero_grad() Build Basic Generative Adversarial Networks week-2 , week-3	6	781	December 17, 2022
Why do we set the retain_graph = True in the discriminators loss? Apply Generative Adversarial Networks week-3	1	583	November 5, 2021
Your First GAN assignment: use of retain_graph=True Build Basic Generative Adversarial Networks week-1	2	757	April 14, 2022
Saving computational graph during discriminator backpropagation Build Basic Generative Adversarial Networks week-1	1	11	July 8, 2024

C1W1 Training Loop

Related topics