I’m currently doing the first assignment of this course (the rnn_cell_forward function so far), and i’m just wondering on what is the optimal number of weight parameters? I don’t remember Andrew talking about the number of weight parameters, but similar to CNNs, I’m guessing this has already been studied by a number of researchers. Anyone knows the number?
Thank you very much!
As with any NN, the number of weight parameters is a compromise between the quality of the results and the amount of time (and computation) required for training.
It’s usually decided through experimentation.
Thank you for the information! But let’s say, I’m building a small network, would 500 weights parameters be a good starting point? Or is it too much?
Thanks again 
There is no specific answer to that question. You didn’t say how many input features you have, for example.
One rule-of-thumb is to start with the square root of the number of input features. Then you have to adjust either higher or lower depending on your specific goals (accuracy vs. training time).
Much clearer. Thank you very much!