C2W4 Lab 2 Multi GPU Mirrored Strategy Loss Object Reduction is NONE rational

Course Q&A TensorFlow: Advanced Techniques Specialization Custom and Distributed Training with TF
1

Hello;

Good Day!

In the C2W4 Lab 2, Multi GPU Mirrored Strategy Loss Object, the code walk through sets Reduction to NONE

What is the rational for the above?

Was it to simply illustrate the concept of manual Loss aggregation across multiple GPUs or is the above a mandatory thing in the case of multiple GPU mirrored strategy scenarios?

Warmest Regards
Steven Sim

2

Hello there,

this link Reduction provides guidance when to use None or else when using strategy.scope().