How to decide the initial value of weight and bias?

Course Q&A Machine Learning Specialization Supervised ML: Regression and Classification
1

Hi there.

For a long time, I’ve been wondering how do we decide the initial value of w (weight) and b (bias) before applying the gradient descent algorithm. Currently I am at week 2 of ML Specialization course 1 and I have observed that before applying gradient descent there is some initial value determined before.

Is it considered randomly or there is some strategy that goes behind it? If I am wrong somewhere kindly correct me :slight_smile:

1 Like
2

Typically the initial weights and biases are set to all-zeros for a simple regression, or to small random values for a neural network.

1 Like
3

Ok got it :slight_smile: Thank you.

1 Like
4

Tom has given us the answer, but here’s a thread which explains why zero initialization works for Logistic Regression, while not for real Neural Networks.