# Number of activation values

**URL:** <https://community.deeplearning.ai/t/number-of-activation-values/211367>\
**Category:** Advanced Learning Algorithms\
**Tags:** week-module-1\
**Created:** [September 28, 2022, 8:14am UTC](https://community.deeplearning.ai/t/number-of-activation-values/211367 "2022-09-28T08:14:01Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Eric5](https://avatars.discourse-cdn.com/v4/letter/e/7feea3/32.png) [@Eric5](https://community.deeplearning.ai/u/Eric5)\
**Post date:** [September 28, 2022, 8:14am UTC](https://community.deeplearning.ai/t/number-of-activation-values/211367/1 "2022-09-28T08:14:01Z")

</div>

Hello

I am a bit confused about the number of activation values in each layer. According to the lecture, in layer 2 with 15 neurons, the output should have 15 activation values. But for first neuron in this 2nd layer  
ie g(w1[2].a[1]+b1[2]) would there not be 25 values just for this neuron as there are 25 activation values ie a[1] fed into this neuron alone

 ![image](https://global.discourse-cdn.com/dlai/original/3X/a/b/abac3cf1bd35b5b8b41662d63736f25776423ddc.jpeg)

Thank you

Eric

---

<div class="post-metadata">

**Author:** ![rmwkwok](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/rmwkwok/32/126977_2.png) [@rmwkwok](https://community.deeplearning.ai/u/rmwkwok)\
**Post date:** [September 28, 2022, 10:07am UTC](https://community.deeplearning.ai/t/number-of-activation-values/211367/2 "2022-09-28T10:07:32Z")

</div>

Hello Eric,

Here’s what’s happening. The first neuron in the layer 2 receives the 25 “activation values” from the layer 1, and each of those “activation values” is multiplied with their corresponding weights in that neuron (i.e. the neuron has 25 w and 1 b). Together with the bias term, the 25 multiplication results are added to become one number which goes to the activation function and become the one and only one “activation value” from the neuron.

Raymond

---

<div class="post-metadata">

**Author:** ![Eric5](https://avatars.discourse-cdn.com/v4/letter/e/7feea3/32.png) [@Eric5](https://community.deeplearning.ai/u/Eric5)\
**Post date:** [September 29, 2022, 2:16am UTC](https://community.deeplearning.ai/t/number-of-activation-values/211367/3 "2022-09-29T02:16:21Z")

</div>

Thanks Raymond for your reply  
So can I clarify if the 25 weights all have different values or are the 25 weights in the same neuron have the same value?

Thank you

Eric

---

<div class="post-metadata">

**Author:** ![rmwkwok](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/rmwkwok/32/126977_2.png) [@rmwkwok](https://community.deeplearning.ai/u/rmwkwok)\
**Post date:** [September 29, 2022, 2:38am UTC](https://community.deeplearning.ai/t/number-of-activation-values/211367/4 "2022-09-29T02:38:19Z")

</div>

They are supposed to be initialized randomly, and then have different values throughout the training process, unless we knowingly overly strongly regularize them that push them to zeros, or we are suffering from some unwanted problems.

Cheers 🙂  
Raymond

PS: [this is a post](https://community.deeplearning.ai/t/how-do-we-get-diff-weights-and-bias-for-a-dense-layer/175851/2) that explains why neurons can diversify and in which there is another link to a post that demonstrates how you can make neurons not to diversify

---

<div class="post-metadata">

**Author:** ![Eric5](https://avatars.discourse-cdn.com/v4/letter/e/7feea3/32.png) [@Eric5](https://community.deeplearning.ai/u/Eric5)\
**Post date:** [October 1, 2022, 8:44am UTC](https://community.deeplearning.ai/t/number-of-activation-values/211367/5 "2022-10-01T08:44:17Z")

</div>

> [@rmwkwok](#):
>
> They are supposed to be initialized randomly, and then have different values throughout the training process, unless we knowingly overly strongly regularize them that push them to zeros, or we are suffering from some unwanted problems.

Thanks so much for that