In the programming assignment, we use the normal vectors to define the hyperplanes which will be used to split the vector space.
The way these vectors are defined, they seem to be position vectors, i.e. all starting at the origin. It would follow that the origin is part of all the hyperplanes. This is different from what I understood from the video’s and e.g. from Fig 4 in the programming assignment, where it seems the hyperplanes are spread all throughout the space and they don’t all meet at the origin.
Which representation is correct? Which representation would lead to the best clustering?
A hyperplane is used to split and divide the original vector space into two parts (positive and negative) based on dot product with its normal vector. When it comes to dot product, the relative positioning does not matter since the normal vector will be moved around to go through the dot product operation with different vectors. Thus I think it does not matter in terms of where they start, including starting at the same origin.
If I understand correctly, each of the splitting planes has 1 normal vector that defines it, where the starting point of each of the normal vectors lies in the plane. When defining the normal vectors as position vectors (as in the assignment), they all start at the origin, as the end (rather than starting) points of the different normal vectors are different / moved around.
The disadvantage of randomly creating planes like this, is that they could never generate a set of planes that separates some points, e.g. point [1,1] and [2,2].
I have attached an image to clarify this for a 2d case. In the left depiction, points [1,1] and [2,2] are inseparable, whereas this is not the case for the right depiction.
