in the course Andrew talk about 1x1 convolution block but i quite dont get it. So i have search more about it and now i am even more confuse. Here’s what i found :
i have researched more and found that 1x1 convolution use less memory, run faster and act as a bottle neck layer to cut down the size of the input for the next layer, am i missing anything ? But i still dont know how can 1x1 convolution can provide non-linearity as the picture above said. How can a 1x1 provide non-linearity but a 3x3 cant ? Furthermore, it cant be the same as a FC layer right ? FC neuron will look at all neuron from previous layer, a 1x1 conv only look at 3 if our previous layer have 3 channel. Thank you for reading
You have posted in the ‘General Discussions’ forum area.
Please move this thread to the forum for the course you are attending.
You can do this using the “pencil” tool in the thread title.
I’m guessing you are asking about DLS C4 (ConvNets) Week 2. Any “conv” layer will provide non-linearity because of the activation function, so the point is not that 3 x 3 can’t provide non-linearity. It’s that it has more parameters. Also note here that the point of MobilNet and the efficiencies it brings is not just the 1 x 1 convolutions, but also the “depthwise separable” convolutions. It’s the combination of the two that give you something approximating the same power as a more traditional “conv” sequence but at much lower cost in terms of memory space and compute. I would suggest that now that you’ve done a bit more thinking and investigation, that you go back and watch the lectures starting at this one and on through the MobilNet lectures again. I bet it will make more sense with what you now know …
