# How do different neurons and layers specialize? **URL:** \ **Category:** Advanced Learning Algorithms\ **Tags:** week-module-2\ **Created:** [November 12, 2022, 4:48am UTC](https://community.deeplearning.ai/t/how-do-different-neurons-and-layers-specialize/229724 "2022-11-12T04:48:45Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![Sashka\_Warner](https://avatars.discourse-cdn.com/v4/letter/s/13edae/32.png) [@Sashka\_Warner](https://community.deeplearning.ai/u/Sashka_Warner)\ **Post date:** [November 12, 2022, 4:48am UTC](https://community.deeplearning.ai/t/how-do-different-neurons-and-layers-specialize/229724/1 "2022-11-12T04:48:45Z")
I have completed the first two courses in the machine learning specialization, and am almost done with the reinforcement learning section of the third course. However, I still don’t understand how different neurons and layers focus on different parts of the input data. For example, in the lesson Example: Recognizing Images, Dr. Ng states that in the first layer one neuron might look for lines oriented in one direction whereas another neuron might look for lines oriented in another direction. Dr. Ng goes on to say that in the second layer the neurons will focus on identifying parts of a face at a larger scale. I understand the basics of forward and back propagation, and I understand how in a convolutional neural network convolutions can be used to detect edges. However, I still don’t understand how the neurons and layers know how to specialize into focusing on different elements. For the neurons, I thought initially that if each neuron is created with different random weights and b values then that could help explain the specialization, but I thought I read somewhere else that you can initialize all neurons with the same weights. If anyone can help me understand how the neurons and layers know to focus on different things I would really appreciate the help! ---
**Author:** ![TMosh](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/tmosh/32/9321_2.png) [@TMosh](https://community.deeplearning.ai/u/TMosh)\ **Post date:** [November 12, 2022, 5:46am UTC](https://community.deeplearning.ai/t/how-do-different-neurons-and-layers-specialize/229724/2 "2022-11-12T05:46:00Z")
The neurons don’t know what sort of pattern they are looking for. The weight values are simply adjusted to minimize the cost. If detecting an edge, or a line orientation, is useful to minimize the cost, that’s just how the weights will turn out. There’s no pre-determined assumption about what the neurons will learn. Andrew is just giving some intuitive context for how you might understand what the NN is doing, and why it gives useful results. ---
**Author:** ![rmwkwok](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/rmwkwok/32/126977_2.png) [@rmwkwok](https://community.deeplearning.ai/u/rmwkwok)\ **Post date:** [November 12, 2022, 10:13am UTC](https://community.deeplearning.ai/t/how-do-different-neurons-and-layers-specialize/229724/3 "2022-11-12T10:13:57Z")
Hello @Sashka_Warner, > [@Sashka\_Warner](#): > > you can initialize all neurons with the same weights. I disagree that initializing neurons with the same set of weights guarantees neurons to capture different features. Please refer to[this post](https://community.deeplearning.ai/t/how-do-we-get-diff-weights-and-bias-for-a-dense-layer/175851/2) for why neurons can differentiate. In the post, there is another link to a post with some mathematical explanations. Raymond ---
**Author:** ![Sashka\_Warner](https://avatars.discourse-cdn.com/v4/letter/s/13edae/32.png) [@Sashka\_Warner](https://community.deeplearning.ai/u/Sashka_Warner)\ **Post date:** [November 12, 2022, 11:01pm UTC](https://community.deeplearning.ai/t/how-do-different-neurons-and-layers-specialize/229724/4 "2022-11-12T23:01:30Z")
Thank you so much for the reply, that’s helpful! I might be missing the point, but if we continue the example of identifying edges and identifying facial components from an image: Is it possible for neuron #1 in Layer 1 to identify edges, while at the same time neuron #2 in Layer 1 identifies facial components? In other words, do neurons in a given layer always work at the same “scale”? ---
**Author:** ![Sashka\_Warner](https://avatars.discourse-cdn.com/v4/letter/s/13edae/32.png) [@Sashka\_Warner](https://community.deeplearning.ai/u/Sashka_Warner)\ **Post date:** [November 12, 2022, 11:04pm UTC](https://community.deeplearning.ai/t/how-do-different-neurons-and-layers-specialize/229724/5 "2022-11-12T23:04:35Z")
Hello @rmwkwok, Thank you very much for the reply! That post you shared that shows the back propagation calculations is extremely helpful! I think that would be a great addition to the lecture slides! Thanks, Sashka ---
**Author:** ![rmwkwok](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/rmwkwok/32/126977_2.png) [@rmwkwok](https://community.deeplearning.ai/u/rmwkwok)\ **Post date:** [November 12, 2022, 11:36pm UTC](https://community.deeplearning.ai/t/how-do-different-neurons-and-layers-specialize/229724/6 "2022-11-12T23:36:41Z")
You are welcome, Sashka! ---
**Author:** ![shanup](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/shanup/32/127654_2.png) [@shanup](https://community.deeplearning.ai/u/shanup)\ **Post date:** [November 13, 2022, 6:54am UTC](https://community.deeplearning.ai/t/how-do-different-neurons-and-layers-specialize/229724/7 "2022-11-13T06:54:43Z")
Hello @Sashka_Warner > [@Sashka\_Warner](#): > > Is it possible for neuron #1 in Layer 1 to identify edges, while at the same time neuron #2 in Layer 1 identifies facial components? Unfortunately, this is not something that we can consciously control. It is the magic of the math that decides all the pieces, so that they can all cohesively contribute to the output at the final layer. We set the learning algorithm on a task to reduce the \frac {dJ} {dw}. We further allow it to apply the chain rule so that J can backpropogate to all the layers. From here on the math takes over. The end result being that different neurons learn different features (edges, parts of the image etc). In this manner, each neuron thereby contributes towards the creation of the final prediction. ---
**Author:** ![TMosh](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/tmosh/32/9321_2.png) [@TMosh](https://community.deeplearning.ai/u/TMosh)\ **Post date:** [November 13, 2022, 7:48am UTC](https://community.deeplearning.ai/t/how-do-different-neurons-and-layers-specialize/229724/8 "2022-11-13T07:48:42Z")
> [@Sashka\_Warner](#): > > In other words, do neurons in a given layer always work at the same “scale”? No guarantees about that. The layers learn whatever will give the minimum cost. ---
**Author:** ![TMosh](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/tmosh/32/9321_2.png) [@TMosh](https://community.deeplearning.ai/u/TMosh)\ **Post date:** [November 13, 2022, 7:49am UTC](https://community.deeplearning.ai/t/how-do-different-neurons-and-layers-specialize/229724/9 "2022-11-13T07:49:33Z")
> [@Sashka\_Warner](#): > > if we continue the example of identifying edges and identifying facial components from an image: It’s not really an example - it’s one of Andrew’s intuitive explanations for a very complicated topic. ---
**Author:** ![Sashka\_Warner](https://avatars.discourse-cdn.com/v4/letter/s/13edae/32.png) [@Sashka\_Warner](https://community.deeplearning.ai/u/Sashka_Warner)\ **Post date:** [November 13, 2022, 6:02pm UTC](https://community.deeplearning.ai/t/how-do-different-neurons-and-layers-specialize/229724/10 "2022-11-13T18:02:09Z")
Hello @shanup , Thank you very much for the reply! Got it - thank you for the help! Best, Sashka ---
**Author:** ![Sashka\_Warner](https://avatars.discourse-cdn.com/v4/letter/s/13edae/32.png) [@Sashka\_Warner](https://community.deeplearning.ai/u/Sashka_Warner)\ **Post date:** [November 13, 2022, 6:03pm UTC](https://community.deeplearning.ai/t/how-do-different-neurons-and-layers-specialize/229724/11 "2022-11-13T18:03:22Z")
Thank you very much for the follow-up reply! I see- thank you for answering my questions! I appreciate it! ---
**Author:** ![Sashka\_Warner](https://avatars.discourse-cdn.com/v4/letter/s/13edae/32.png) [@Sashka\_Warner](https://community.deeplearning.ai/u/Sashka_Warner)\ **Post date:** [November 13, 2022, 6:03pm UTC](https://community.deeplearning.ai/t/how-do-different-neurons-and-layers-specialize/229724/12 "2022-11-13T18:03:56Z")
Got it - makes sense! Thanks!