C2_W2_Neural Network from Scratch

Course Q&A Machine Learning Specialization Advanced Learning Algorithms
1

In the video on Training details of TensorFlow implementation,
Prof. Andrew Ng mentioned it is beneficial to know how to implement
NN models from scratch and not merely use libraries like TF & PyTorch.

Where can I learn to do this? Any good book or online resource for proceeding?

2

Apart from books that you can possibly find, you can take the Deep Learning Specialization!

3

Definitely, as @gent.spah mentioned. Deep Learning Specialization is the way to go. There you will at the beginning of the course a fully connected network from scratch.

4

Thank you.
I will look into it after finishing the ML Specialization which I’m currently enrolled in.

5

Each DLS course has you implement one NN system by hand, then it switches to using TensorFlow for the rest of that course.

6

How to choose the number of neurons(logistic regressions) in a layer? For ex, in that predicting the number 0 or 1, they used 25 neurons in first layer, then 15 in second and 1 in the last. How to figure this out?

7

Hi @Karika_Patel one way would be to plot accuracy vs number of neurons vs number of layers is expensive but precise.

8

The size of the output layer is easy - it’s the number of labels or classes (or for true/false, just one output).

There’s no hard rule for setting up the number of hidden layers or number of units per layer. It’s more a matter of experience and experimentation.

  • The more hidden layers you add, the more computations will be necessary during training.
  • The more units per layer, the more complex will be the model - which can lead to overfitting.

Ideally you strike a balance so you have just enough complexity to get just good-enough results.

The instructors for this assignment picked 25 and 15 because it demonstrates what they’re aiming to teach.