Hi, I was wondering about something far beyond the scope of the courses here. It is probably something more suitable for a PhD thesis, but I thought I would ask.
Disclosure: I am not a physicist, I have average understanding of linear algebra (thanks to the great series by 3Blue1Brown - link below), I am a beginner in machine learning.
So my question is:
Even when using non-linear equations as activation functions, machine learning is still carried out in the realm of linear algebra, right? (my understanding so far)
So geometrically, all the transformations change the “shape” of the space in a linear fashion. So if space is represented by a grid of vertical and horizontal lines, the transformations will change the grid such that the angles between the intersections are potentially no longer right angles, but the lines of the grid are all still straight lines.
What happens when you start considering non-linear algebra as is done in advanced physics to explain, for example, gravity warping space. Would entering this realm of non-linear algebra allow machine learning models to capture even more non-linearity or would the models just break down?
3Blue1Brown: https://www.youtube.com/watch?v=kjBOesZCoqc&list=PL0-GT3co4r2y2YErbmuJw2L5tW4Ew2O5B