# Week 3 Random Initialization **URL:** \ **Category:** Neural Networks and Deep Learning\ **Tags:** coursera-platform\ **Created:** [January 11, 2022, 11:02pm UTC](https://community.deeplearning.ai/t/week-3-random-initialization/85674 "2022-01-11T23:02:03Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Kong\_Xiangwei](https://avatars.discourse-cdn.com/v4/letter/k/8491ac/32.png) [@Kong\_Xiangwei](https://community.deeplearning.ai/u/Kong_Xiangwei)\ **Post date:** [January 11, 2022, 11:02pm UTC](https://community.deeplearning.ai/t/week-3-random-initialization/85674/1 "2022-01-11T23:02:03Z")
In the lecture, Prof. Andrew suggests that we should randomize the parameters W and b. Then he mentioned that we need to make sure that W parameters not all be 0, but b can be zero. I agree. However, I am wondering why we cannot make all b to be not zero, but make W to be zero. Won’t this also work? ---
**Author:** ![paulinpaloalto](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/paulinpaloalto/32/187_2.png) [@paulinpaloalto](https://community.deeplearning.ai/u/paulinpaloalto)\ **Post date:** [January 12, 2022, 12:05am UTC](https://community.deeplearning.ai/t/week-3-random-initialization/85674/2 "2022-01-12T00:05:38Z")
Yes, you can do it either way and still “break symmetry”. But the common practice is to randomly set the W values and zero the b values. My guess is that this gives you faster learning but this is an experimental science: you can try it both ways and see what happens. Well, I guess there are three different methods to compare: random W and zero b, zero W and random b and then both of them random. [Here’s a thread](https://community.deeplearning.ai/t/symmetry-breaking-versus-zero-initialization/16061) which goes into a bit more detail about Symmetry Breaking and mentions the issue that you point out. ---
**Author:** ![paulinpaloalto](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/paulinpaloalto/32/187_2.png) [@paulinpaloalto](https://community.deeplearning.ai/u/paulinpaloalto)\ **Post date:** [January 12, 2022, 12:09am UTC](https://community.deeplearning.ai/t/week-3-random-initialization/85674/3 "2022-01-12T00:09:59Z")
It is also worth pointing out that the initialization algorithm turns out to be an important “hyperparameter”, meaning a design choice that you need to make. There are a large number of different algorithms that have been developed, but no one “silver bullet” solution that works the best in all cases. So doing the experiment suggested above on one particular dataset and model is just one data point in a very large search space. This is a more advanced topic that Prof Ng does not have time to cover here in Course 1. We are just getting started and there is a lot more to say. He will go into more detail on initialization in Week 1 of Course 2, so please stay tuned for that. ---
**Author:** ![skrichen](https://avatars.discourse-cdn.com/v4/letter/s/f0a364/32.png) [@skrichen](https://community.deeplearning.ai/u/skrichen)\ **Post date:** [May 6, 2022, 8:05pm UTC](https://community.deeplearning.ai/t/week-3-random-initialization/85674/4 "2022-05-06T20:05:27Z")
Prof. Ng mentioned that W should not all be 0 because, if we are not breaking the symmetry, then the hidden units will be symmetric as they will be computing the same functions. But doesn’t this mean that W should not be symmetric as a starting point? From my understanding it shouldn’t also be all 1 or all 2 as well? ---
**Author:** ![paulinpaloalto](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/paulinpaloalto/32/187_2.png) [@paulinpaloalto](https://community.deeplearning.ai/u/paulinpaloalto)\ **Post date:** [May 6, 2022, 8:41pm UTC](https://community.deeplearning.ai/t/week-3-random-initialization/85674/5 "2022-05-06T20:41:52Z")
Exactly. That is why we randomly initialize all the W values. ---
**Author:** ![paulinpaloalto](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/paulinpaloalto/32/187_2.png) [@paulinpaloalto](https://community.deeplearning.ai/u/paulinpaloalto)\ **Post date:** [May 6, 2022, 9:09pm UTC](https://community.deeplearning.ai/t/week-3-random-initialization/85674/6 "2022-05-06T21:09:43Z")
Note that I gave a link in one of my earlier replies on this thread to a thread that goes into more detail about symmetry breaking. ---
**Author:** ![skrichen](https://avatars.discourse-cdn.com/v4/letter/s/f0a364/32.png) [@skrichen](https://community.deeplearning.ai/u/skrichen)\ **Post date:** [May 6, 2022, 10:39pm UTC](https://community.deeplearning.ai/t/week-3-random-initialization/85674/7 "2022-05-06T22:39:36Z")
I checked the link, and your response was very thorough. Thank you for pointing it out to me.