I did not take the Math4ML course, so I do not know the course specific context. But I still hope to contribute to answer your question:
As some physics motivation:
In structural dynamics, when you encounter a specific structure, it reacts after excitement (e.g. hitting a steel beam with a hammer): in general after the system is excited, it usually oscillates at the natural frequency of the system (= eigenfrequency) which can be characterised by eigenvalues. The modal forms of the vibration are described by the eigenvectors, which so to speak describe the direction of vibration in the space.
So, in general eigen value problems are of particular benefit if linear systems are analyzed or modelled - also in data analysis of course!
- you can conduct a transformation on the eigenvectors or a subset of them (meaning you change the basis of the original coordinate system to the eigenvector coordinate system [which results in uncoupled equations - called diagonal matrix, see @lucas.coutinho’s excellent response - in structural dynamics which is nice for analysis purposes]). In the transformation case you will do a linear projection which can mean: stretching, scaling etc.
- a transformation (e.g. in our example called modal transformation) is in fact a linear operation, see also this thread: Does embedding projector use dimensional reduction? - #4 by Christian_Simonis
- often is it used in the course of a PCA / SVD to reduce the dimensions of the feature space, see also this thread here: C3_W2 - PCA Question Often you can dramatically speed up the calculation of linear (but also linearized nonlinear) problems (like contact mechanics in structural dynamics) because here inverting a matrix (iteratively) is needed which has a cubic complexity. If you can reduce the matrix with a PCA / SVD only using the most important eigen vectors / eigen values describing most of the information (e.g 98% or so) , you barely loose any accuracy in the linear or linearised system model but you are much faster - mainly due to the model order reduction aspect when it comes to matrix inverting step.
Hope that helps!
Best regards
Christian