Perplexed with `strategy.reduce()` method:

hi
I got lost in terms of the meaning of strategy.reduce and what it does.
Here is a relevant code from the lab 3.

strategy.reduce(tf.distribute.ReduceOp.SUM, per_replica_losses, axis=None)

Andrew explains somewhere in the lecture the ‘map reduce’: it does aggregate losses.
Does strategy.reduce do the same thing as ‘map reduce’?

If so, I think that it is a very confusing name: why is this method called ‘reduce’, if it aggregate somethings?
If not, please advise me what ‘reduce’ does in the case of strategy.reduce?

I would appreciate it if anyone can do mentoring on this.
Thanks
Best regards

@Michio_Suginoo, you’re exactly right. The term “reduce” comes from the terminology used to describe the “map reduce” approach for working with distributed data. The general idea is that “mapping” is used to help us determine how to distribute the data across multiple processors that will be working in parallel. Then the “reduce” part will aggregate the work of all those processors.

The terms have been around for a while, and may not be the best, but I think the concept behind using the word “reduce” comes from the idea of “reducing” the set of parallel processes back down to one.

1 Like

Thanks @Wendy for your note.
Now, I understand the historical background the issue.
Thanks
Kind regards,
Michio