Hi Matthias,
thanks for your open feedback!
I can certainly understand your thoughts. In the end CNN architecture and layers are really enhanced and it can feel a certain magic is happening here.
Still, let me try to demystify it at least to a small extend: convolution is a really well established and recognised signal processing method in control and system theory, see also this thread: How to Calculate the Convolution? - #2 by Christian_Simonis. So when you come rather from a Mechatronics Engineering world (like I do) and used to apply autocorrelation, do some audio or image processing, apply Laplace or Fourier transform, time series analysis and filtering, it rather seems a logical step to incorporate this filter option into layer architectures and allow the net to learn parameters on its own using tons of data.
You are also right that interpretability of complex architectures might be rather limited. Still there are possibilities, like heat map analysis, see also this thread: What makes the different neurons in a layer calculate different parameters? - #7 by Christian_Simonis
[ One side note: convolution also plays an important role in probability theory and also provides a nice illustration why so many distributions in reality are normally distributed.
- E.g. if you would convolute two uniform distributions, you will get a triangle distribution (like in the distributions of the sum of points on two dices)
- if you keep adding more uniform distributions (corresponding to the sum of points on three (4,…) dices), you will approach gradually a normal distribution shape.
So convolution is really well established in many mathematical core concepts.
]
I would be interested, do you have suggestions if some real world applications of 1D signal processing with convolution would have helped here… what do you think, @Matthias_Kleine?
Best regards
Christian