The notes in the week 1’s first programming assignment state that initializing the weights to zero prevents the neural net from breaking symmetry. It thus becomes like a logistic regression. When the code is run, the loss does not decrease. So logistic regression loss will not decrease if weights ar…

Course 2 Week 1 PA 1: Why does zero weight cause no change in loss

paulinpaloalto January 5, 2022, 10:15pm 2

As always, it goes back to the math. The math is different in the two cases. Here’s a thread which explains this in some detail.

1 Like