# Neural Network Clarification

**URL:** <https://community.deeplearning.ai/t/neural-network-clarification/744785>\
**Category:** Advanced Learning Algorithms\
**Tags:** week-module-2\
**Created:** [January 2, 2025, 5:53pm UTC](https://community.deeplearning.ai/t/neural-network-clarification/744785 "2025-01-02T17:53:17Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![satmuri](https://avatars.discourse-cdn.com/v4/letter/s/ee59a6/32.png) [@satmuri](https://community.deeplearning.ai/u/satmuri)\
**Post date:** [January 2, 2025, 5:53pm UTC](https://community.deeplearning.ai/t/neural-network-clarification/744785/1 "2025-01-02T17:53:17Z")

</div>

Say we are dealing with a logistic cost that is a convex function, I am confused how Neural Networks in each layer though initialized with different parameters, how come each neuron does not come up with the same output after gradient descent? I don’t understand how the neural network doesn’t collapse, and I feel like I am missing some intuition of how the neural networks learn patterns by themselves.

---

<div class="post-metadata">

**Author:** ![TMosh](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/tmosh/32/9321_2.png) [@TMosh](https://community.deeplearning.ai/u/TMosh)\
**Post date:** [January 2, 2025, 6:27pm UTC](https://community.deeplearning.ai/t/neural-network-clarification/744785/2 "2025-01-02T18:27:47Z")

</div>

Because of the non-linear activation function in the hidden layers, the NN cost function is not convex.

Given different initial values, each weight will follow its own trajectory to a value that minimizes the cost.

It seems like magic, and I don’t have a mathematical proof. But it works.

---

<div class="post-metadata">

**Author:** ![SNaveenMathew](https://avatars.discourse-cdn.com/v4/letter/s/a587f6/32.png) [@SNaveenMathew](https://community.deeplearning.ai/u/SNaveenMathew)\
**Post date:** [January 3, 2025, 11:00pm UTC](https://community.deeplearning.ai/t/neural-network-clarification/744785/3 "2025-01-03T23:00:37Z")

</div>

Your intuition is right under special circumstances where the gradients wrt each weight in a layer will be identical if all the weights are initialized the same constant. This Stack Overflow discussion gives you some more intuition.

> <https://stackoverflow.com/questions/20027598/why-should-weights-of-neural-networks-be-initialized-to-random-numbers>