why there’s no some kind of algorithm or a mathematical way to determine the appropriate number of filters, layers … in a deep NN ?
or is there a way but I didn’t get it ? what am I missing ?
It’s possible that someone will someday figure out an algorithmic way to determine appropriate values for such “hyperparameters”, but I have not heard of any such technology that exists yet. Have you taken DLS Course 2 yet? That is the place I have seen where Prof Ng discusses how to go about choosing such hyperparameters in a systematic way.
thank you for answering
yes I’ve completed the whole specialization
but I was asking if such an algorithm exists and if not, is it because of some kind of mathematical/computational boundary of just we didn’t get there yet ?
so this is means that it maybe exists but we just don’t know it yet
1 Like
Think of all the different types of networks there are (DNN, CNN, RNN) and all the degrees of freedom there are in the architectures of those networks. Now think of how you would characterize the problem you are trying to solve in an algorithmic way. You’re basically asking for an algorithm that can convert a description of a problem into a solution for that problem. How would you algorithmically characterize the functionality of the computer vision system you would need for an autonomous vehicle?
At the mathematical level, you have the Universal Approximation Theorem, which makes general statements about the existence of classes of functions with particular behaviors, but it is the classic “existence proof”: it tells you that such a thing exists, but gives you no help in actually finding it.
I’m not an expert in any of this stuff, so I’m the wrong person to answer this question. Just throwing out some ideas here. All I know on this is what I’ve heard Prof Ng say in these various courses.
At a high level, if such an algorithm existed, I think it’s a pretty sure bet that we would have heard about it by now. So I think it’s very unlikely that such an algorithm currently exists. Then the big question is whether it is theoretically possible in the future or not. I have no idea how to answer that question. 
2 Likes
yes exactly this is the question I had in mind, I didn’t find any research tried to answer this question.
really thank you for your answer.
1 Like