Inception-Net Backprop order question


I searched some threads and I think I understand that the intermediate side branches provide loss functions which help ensure that the weights of the network as it gets deeper don’t get out of whack.

My question is, in what order does back prop happen?

Is it.

  1. Add up final loss function + intermediate loss functions than do a single back prop from the final output?

  2. Do individual backprops from each intermediate side branch and final loss function?

2a If individual backprops, do you start at first side branch or the final output as the first backprop in turn. If so, do the earlier layers in the network effectively undergo backprop multiple times per pass?

Hello @John_Pan,

It is number one. There is finally only one loss function which is the sum of those contributing factors, and back prop is done to minimize that loss function.


Thank you.