C2W4 Lab 2 Multi GPU Mirrored Strategy Loss Object Reduction is NONE rational


Good Day!

In the C2W4 Lab 2, Multi GPU Mirrored Strategy Loss Object, the code walk through sets Reduction to NONE

What is the rational for the above?

Was it to simply illustrate the concept of manual Loss aggregation across multiple GPUs or is the above a mandatory thing in the case of multiple GPU mirrored strategy scenarios?

Warmest Regards
Steven Sim

Hello there,

this link Reduction provides guidance when to use None or else when using strategy.scope().