# Layer 2 Weights

**URL:** <https://community.deeplearning.ai/t/layer-2-weights/439967>\
**Category:** Advanced Learning Algorithms\
**Tags:** week-module-1\
**Created:** [September 8, 2023, 10:16am UTC](https://community.deeplearning.ai/t/layer-2-weights/439967 "2023-09-08T10:16:07Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Amit\_Misra1](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/amit_misra1/32/48206_2.png) [@Amit\_Misra1](https://community.deeplearning.ai/u/Amit_Misra1)\
**Post date:** [September 8, 2023, 10:16am UTC](https://community.deeplearning.ai/t/layer-2-weights/439967/1 "2023-09-08T10:16:07Z")

</div>

I’m trying to replicate the practice lab in Google Colab. When I print the layer 2 weights, my question is, where do those weights come from when the model hasn’t even been trained yet?

Also, are the weights supposed to be different than the practice lab? See screenshot below of the weights for layer 2 on the practice lab on the left and my reproduction of it on the right.

 ![image](https://global.discourse-cdn.com/dlai/original/3X/b/d/bd8c5b9b51c00874f9d7de9af22d6ad6de2d133d.png)

---

<div class="post-metadata">

**Author:** ![gent.spah](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/gent.spah/32/136423_2.png) [@gent.spah](https://community.deeplearning.ai/u/gent.spah)\
**Post date:** [September 8, 2023, 11:11am UTC](https://community.deeplearning.ai/t/layer-2-weights/439967/2 "2023-09-08T11:11:43Z")

</div>

Even if the model hasn’t started training there should be some random initialization of weights at every layer and I think because no seed is being used its not the same as in the practice lab.

---

<div class="post-metadata">

**Author:** ![Amit\_Misra1](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/amit_misra1/32/48206_2.png) [@Amit\_Misra1](https://community.deeplearning.ai/u/Amit_Misra1)\
**Post date:** [September 8, 2023, 11:53am UTC](https://community.deeplearning.ai/t/layer-2-weights/439967/3 "2023-09-08T11:53:46Z")

</div>

Thanks for the response. Where are those initial weights coming from?

---

<div class="post-metadata">

**Author:** ![gent.spah](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/gent.spah/32/136423_2.png) [@gent.spah](https://community.deeplearning.ai/u/gent.spah)\
**Post date:** [September 8, 2023, 12:02pm UTC](https://community.deeplearning.ai/t/layer-2-weights/439967/4 "2023-09-08T12:02:54Z")

</div>

> [@gent.spah](#):
>
> there should be some random initialization of weights at every layer

Search in google random initialization of NN weights.

---

<div class="post-metadata">

**Author:** ![Amit\_Misra1](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/amit_misra1/32/48206_2.png) [@Amit\_Misra1](https://community.deeplearning.ai/u/Amit_Misra1)\
**Post date:** [September 8, 2023, 12:17pm UTC](https://community.deeplearning.ai/t/layer-2-weights/439967/5 "2023-09-08T12:17:26Z")

</div>

Is it built into Tensor/Keras to determine those initial weights?

---

<div class="post-metadata">

**Author:** ![TMosh](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/tmosh/32/9321_2.png) [@TMosh](https://community.deeplearning.ai/u/TMosh)\
**Post date:** [September 8, 2023, 4:58pm UTC](https://community.deeplearning.ai/t/layer-2-weights/439967/6 "2023-09-08T16:58:23Z")

</div>

Yes. TensorFlow initializes the weights as soon as an instance of the layer is created.

---

<div class="post-metadata">

**Author:** ![Amit\_Misra1](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/amit_misra1/32/48206_2.png) [@Amit\_Misra1](https://community.deeplearning.ai/u/Amit_Misra1)\
**Post date:** [September 8, 2023, 5:30pm UTC](https://community.deeplearning.ai/t/layer-2-weights/439967/7 "2023-09-08T17:30:30Z")

</div>

Awesome, thanks for the info!