I’m trying to replicate the practice lab in Google Colab. When I print the layer 2 weights, my question is, where do those weights come from when the model hasn’t even been trained yet?
Also, are the weights supposed to be different than the practice lab? See screenshot below of the weights for layer 2 on the practice lab on the left and my reproduction of it on the right.
Even if the model hasn’t started training there should be some random initialization of weights at every layer and I think because no seed is being used its not the same as in the practice lab.
Thanks for the response. Where are those initial weights coming from?
Search in google random initialization of NN weights.
Is it built into Tensor/Keras to determine those initial weights?
Yes. TensorFlow initializes the weights as soon as an instance of the layer is created.
Awesome, thanks for the info!
