When do we use inplace=True for nn.ReLU?

Course Q&A Generative Adversarial Networks (GANS) Build Basic Generative Adversarial Networks
1

In homework 1,

Why did we do inplace=True for the generator block? but not the discriminator block?

2

Hi River-Blue, welcome to the community!

Well, in the generator block, the instructors have already included nn.ReLU(inplace=True) as part of pre-written code and in the generator block they have asked learners to include Leaky ReLU — That’s the difference (You can definitely use inplace parameter over there).

Typically setting the inplace parameter as True means that it will modify the input directly, without allocating any additional output, and since we are using sequential API over there, it makes a lot of sense as we cannot assign the value to different variables in between.

Hope you get the point, if not feel free to post your queries.

Regards,
Nithin

3

So it’s okay whether to do in place or not? I guess my question is, does the function require us to modify the input itself or return something? Do we actually need inplace=True?

5

Normally people do this to save memory, if memory is not an issue I would say you do not need at all inplace=True

6

thank you!

1 Like