Cats Classifier using One Hidden Layer

Course Q&A Deep Learning Specialization Neural Networks and Deep Learning
1

Hello,

Out of curiosity, I thought why not implement a Cats Classifier, as we had done in Week 1 using Logistic Regression, using the One Hidden Layer approach taught during Week 3. I was expecting that there must be some improvement in the train and test accuracy. Turns out otherwise…

  1. With 2k iterations and lr=0.005
    image
    image1320×669 56 KB
  2. With 10k iterations and lr=0.009
    image
    image1318×252 22.6 KB

    Also note that Logistic Regression took 38 seconds for this case while One Hidden Layer Neural Network took 65 seconds (almost double). So not only is the test accuracy slightly less, but it is taking more time.
    I wanted to ask, is this expected?

Thanks,
Sushant.

2

There is no guarantee that an arbitrary NN solution will necessarily work better than LR in a given situation. But notice that the architecture they show in the Week 3 single hidden layer case is different in some ways than what you will see in the general case in Week 4. In particular, notice that they use tanh for the hidden layer activation here, but next week Prof Ng switches to using ReLU as the hidden layer activation. You can try several experiments:

  1. Maybe 4 neurons in the hidden layer is not adequate for detecting cats as opposed to the simpler geometry problem that they’re solving in Week 3. Try bigger numbers of neurons and see if that changes the results.
  2. Try solving the actual week 3 problem using ReLU instead of tanh. Note that you’ll also have to change the backprop logic to get that to work. You’ll find that you need a lot more neurons with ReLU even for the simpler problem.
  3. Then try using your ReLU version to solve the “is this a cat” problem. Or better yet, just wait until you get to Week 4 and the assignments have us construct several solutions including 1 hidden layer and 3 hidden layers for the cat classifier. By the end, you’ll have the fully general mechanisms and can experiment with more layers as well. The actual solutions they have us build will be better in both cases than the LR solution from week 2 and you can then try to beat their solutions by experimenting further.
3

Sure. I think it is better for me to get to week 4. At least am relieved that it is just normal and not a bug again! :smile:
Also, I did try more number of nodes for the hidden layer, but that actually worsens the accuracy. So maybe better I go ahead with Week 4 first!

1 Like