Course 2 Week 1 PA 1: Why does zero weight cause no change in loss

The notes in the week 1’s first programming assignment state that initializing the weights to zero prevents the neural net from breaking symmetry. It thus becomes like a logistic regression. When the code is run, the loss does not decrease. So logistic regression loss will not decrease if weights are initialized to zero.

However, in course 1 we initialize our weights in logistic regression to zero, and we get a decreasing loss.

Why is this?

As always, it goes back to the math. The math is different in the two cases. Here’s a thread which explains this in some detail.

1 Like

Thank you, that makes sense!