Few questions about CNN

Hi Mentors and friends,

Q1: in the pic above. The filter is W [1] is the parameters needed to be trained if this filter is not given (if my understanding is correct). So, my question is, if this is given sobel edge filter, do we still need to train it ? I assume it’s given already, those parameters are there already, so no need to train. Is this correct ?

Q2: in the pic above, sharing and sparsity are all talking about one thing: less number of parameter training. Is this correct ?

Thank you!

When you’re training a system, you don’t pre-define what the filters are. They will be learned from the training set.

If an edge detection is useful for this data set, then one will be learned.

If you were doing image processing (not machine learning), you might decide on your own that an edge detector might be useful.

For your second question, those are reasons why a convolution can be useful.

1 Like

May I confirm my understanding as follow:

  1. In CNN, all filters are parameters needed to be trained. They are not pre-defined at all. Hence, if one edge detector is needed, this edge detector will be learned after training. Am I right ?

  2. When you say "image processing ", that is not CNN things, right? You just want to detect the edge, so you just use it directly. No “learning” for the fliter at all. Am I right?

  3. for Q3, I was not clear about my question. Allow me to clarify it.
    Q3_1: since you are all using the one filter to compute the output (sharing), so… then what? why sharing is good?
    Q3_2: so why sparsity is good ? If each output value depends only on a small number of input, why this is good?

  1. Yes, in the kind of networks Prof Ng shows us here, all the filters are learned, rather than predefined. If an edge detector is useful, that will be discovered (“learned”) during training by using back propagation to reduce the error.

  2. There are some cases in which it can be useful to “preprocess” images with known image processing techniques like sharpening or resizing or the like. That could include predefined convolutional filters, but that would not really be part of the trained CNN. It would be a separate standalone preprocessing step.

  3. Sharing is one of the fundamental things about ConvNets that is good: the filters give a much more efficient way to detect patterns that may occur at many points in the image. Think about the contrast with the Feed Forward Nets that we learned about in Course 1: there it only looks at the entire image at once. “Sparsity” is just another way to look at the sharing aspect just described: it means you can efficiently find patterns that are localized and may be repeated in multiple locations within the image.

1 Like

I would express this as the edge detector will be learned during training. Learning, aka iteratively adjusting the filter weights while minimizing loss or error, is what training is. If a filter with the characteristics of edge detection is what minimizes error, then that is what will be learned.

I hope @paulinpaloalto finds this consistent with his #1 above

1 Like