Hello everybody, I see that in the assignment where we build a NN from scratch, we randomly initialize the weights and then we multiply them by 0,01. I understand the randomization: it’s to break the symmetry. I don’t understand why we multiply them by a small number (0.01). Is it to make the wei…

Why do we multiply the random intial weights by 0.01?

jenitta.jebaraj September 2, 2022, 12:59pm 2

Hello @Riccardo_Andreoni
we are multiplying the initial weights by 0.01, so that Z initially places near a good value of g(Z).
For more details kindly check Symmetry Breaking versus Zero Initialization
regards
Jenitta

Topic		Replies	Views
Can the random initialization of weights return very small values using np.random.randn((x,y))*0.001? Neural Networks and Deep Learning coursera-platform	3	692	September 28, 2021
Why random weights help in faster convergence? AI Discussions	8	79	January 18, 2023
Confusion about `Exercise 2 - initialize_parameters_random`: why small initiations cost bad performance Improving Deep Neural Networks: Hyperparameter tun week-module-1 , coursera-platform	2	16	March 17, 2025
Randomly initialize parameter b instead of W Neural Networks and Deep Learning coursera-platform	6	660	August 23, 2022
Why do we select randn instead of rand and why do we multiply by 0.01? Neural Networks and Deep Learning week-module-3 , coursera-platform	2	288	January 15, 2024