How to decide the initial value of weight and bias?

Hi there.

For a long time, I’ve been wondering how do we decide the initial value of w (weight) and b (bias) before applying the gradient descent algorithm. Currently I am at week 2 of ML Specialization course 1 and I have observed that before applying gradient descent there is some initial value determined before.

Is it considered randomly or there is some strategy that goes behind it? If I am wrong somewhere kindly correct me :slight_smile:

1 Like

Typically the initial weights and biases are set to all-zeros for a simple regression, or to small random values for a neural network.

1 Like

Ok got it :slight_smile: Thank you.

1 Like

Tom has given us the answer, but here’s a thread which explains why zero initialization works for Logistic Regression, while not for real Neural Networks.