How to decide the hidden layer size


Hi everyone,

I tried to finish the last assignment but this really concerns me. I do not really understand how to decide the size of hidden layer as it seems like programmer can adjust it within reasonable range.

I simply put n_h = 2 and tests are all passed. I just cannot …

Warm regards
Jimmy

1 Like

The size of the hidden layer is determined by experimentation. It’s a design decision.

  • You want a complex enough model to get good results.
  • But you don’t want it so complex that it takes too long to train, uses too much memory, etc.
3 Likes

Hi Jimmy,

The general rules are like what Tom has shared. In machine learning, we learn how to diagnose variance and bias problem, base on which we may shrink or expand our networks (should also be noted that changing network size isn’t the necessary or only way).

However, as for this particular exercise, even though I don’t know about it as I don’t mentor this course, it should come with description on how to work on it? Didn’t the exercise hint you to use any particular value for the hidden layer? I guess it has :wink:

Cheers,
Raymond

3 Likes

Hi @Jimmy0Jimmy it is perfectly fine to not understand everything at the end of a course, this is a sign that you are pushing beyond your knowledge and it will be an opportunity to grow and learn.

In this case, as @TMosh points out it is mostly on experimentation and trial and error. If you don’t understand something you can go back the lectures and redo the assignments but also you can search online for different examples and try some yourself it will help you yo build a strong foundation.

I hope this helps

2 Likes