Doubt in concept of neural network representation (week3 video3)

The point is what I said earlier: LR is just the output layer of the NN, so it has to have one output, which is the “yes/no” classification result, right?

In the earlier layers of a Neural Network, the power comes from the fact that you can have lots of outputs from each of the layers. That enables the network to learn to detect lots of different things in the inputs: each neuron will be “specialized” during back propagation because of Symmetry Breaking. The whole point is that you don’t want them all to learn the same thing. E.g. in the case of images, the inputs are incredibly complicated and contain lots of different shapes, edges, curves, colors. You need to be able to detect lots of different low level features and then put those together in the later layers to recognize more complex features like a cat’s ear or tail or whiskers.

If each layer only had one output, then there’s really not a lot of point in having multiple layers. Draw the picture by analogy with the network diagrams that Prof Ng shows. Not very interesting with only one output per layer, right?

1 Like