Indeed, good point!
In addition, @LukeNukem:
Also when it comes to the PCA transformation (to the PC’s spanned space) you do a matrix multiplication.
The key point of dimensionality reduction is to find the right balance between:
- enhancing calculation speed (since you got rid of redundant information and operate in a more compact space)
- and the loss of information / accuracy, see also this post here: C3_W2 - PCA Question - #3 by Christian_Simonis and this plot:
While reducing the dimensional space:
- one can improve computational efficiency (one can solve the problem in a smaller dimensional space and often you need to invert a matrix in a smaller space which is cubic effort and this especially means a strong benefit for non-linear problems which you need to solve iteratively which means in total you get much faster of course)
- but you sacrifice accuracy or let’s say you lose some information in this dimensionality reduction.
So there is no free lunch I am afraid.
Best regards
Christian