We are struck. We would like to understand why the need of preferring CNN than regular ANN ?
As per our understanding, Regular NN for large image needs to learn more number of weight which makes overfit problem as well as computational cost is high. Thats why proffesor told need to go for CNN. But Proff Andrew Ng didnt speak about in the First video week1 (Computer Vision) that is how CNN going to reduce weights & good to use when large image given as input ? At which video will be covered ?
Can the regular ANN wont be able to capture if image is in different position like left corner, right corner, top corner ? Is it not possible for Regular ANN ? Is it okay to try Regular NN train the model more hidden units & hidden layer to capture the image in different translation ? Only CNN can capture it ?
Are we loosing spatial properties in Regular NN ? CNN capturing spatial properties (location information) really ? Some people talk about this ? But im not sure is this statement true ?
Is there any difference from CNN over ANN ? Dear Mentor please share ur thoughts super benefits of CNN than ANN?
To be fair, I think Prof Ng did cover all this in the lectures. So maybe you didn’t listen hard enough the first time and perhaps it would be a good idea to watch them again.
Think back to how we processed the images in Course 1 in order to use them as input to our ANNs: we took the 3D tensors and “unrolled” or flattened them into vectors. When you do that, the geometric relationships between the pixels are not exactly “lost”, but they are changed into a form in which it’s more difficult to “see” them. We can see from the performance of the networks in Course 1 that the ANNs can still learn those patterns, but they don’t do as good a job of it as a ConvNet can do. In fact you can do the unrolling in at least two distinct ways and the DNNs can learn the patterns either way. See this thread for more discussion of that point (read down to the posts later in the thread which discuss the “F” order). In the CNN case, we can just leave the images as the 3D tensors that they are with the geometric relationships intact. The CNN naturally steps the filters through the image in a geometric way and the same filter is applied at all the locations in the image (subject to the stride and filter size of course).
An ANN can also learn to find patterns that may be located in different parts of an image, but the CNN is naturally suited to doing that with fewer weights. To do the “number of parameters” comparison, just think about the first layer in the two cases. Let’s just use the cat image dataset from Course 1 Week 4 and the L layer model there as the test case for comparison. Those are 64 x 64 x 3 images with a total of 12288 pixels in each one. The first hidden layer of the ANN in that case was defined to have 20 output neurons. So W^{[1]} will be 20 x 12288 and b^{[1]} will be 20 x 1. So that’s a total of 245,780 parameters in that first hidden layer. In the CNN case, it depends on the filter size we chose and the number of output channels, but say we choose 5 x 5 filters and 64 output channels. That gives us:
5 * 5 * 3 * 64 + 64 = 4,864
parameters. Big difference. Now you can argue that this is not really a fair comparison, since the networks may not have the same number of layers, but it gives you a sense for what Prof Ng means when he says that CNNs typically have fewer parameters to train. Of course if you stick with this and watch all the lectures of Week 1, you’ll see Prof Ng present some typical CNN architectures for doing image classifications and they typically have some Fully Connected layers at the end after a series of Conv and Pooling layers.
Rather than going deeper down this rat hole right at the moment, my suggestion is that you “hold that thought” and continue with Course 4. By the time you get to the end, you will have seen lots of examples of different CNN architectures and also the very interesting lecture titled “What are Deep ConvNets Learning” in Week 4.