Dense function scratch implementation

In one of the videos’ we saw how we implement a neural network from scratch. But regarding this function we created called ‘dense’ how do we produce the activations of this particular layer if the model hasn’t been trained, since the weights won’t be defined yet. Do we produce random activations before training?

Hi @Srivaths_Gondi

In the next week the course will take about how to set activation function with tensorflow and in the deep learning specialization will learn about how to set activation function in every layer from scratch

Please feel free to ask any question,
Thanks,
Abdelrahman

Hello @Srivaths_Gondi

If you meant activations as in ReLU etc, Dense() has default activation of None. Thiis is equivalent to a linear activation. i.e. output = input.

You can change it by setting the “activation” argument in Dense()

If you meant weights - Yes, the weights are randomly initialized before the training starts. So if you print out the weights before and after training, they will be different.