Is there a connection between random planes and Support Vector machines?
Each plane in the set of random planes feels very SVM-like. Do they belong to some greater family of algorithms?
I am not sure what do you mean by random planes. Considering them as same as hyperplanes.
The initial idea of SVM is an extension of hyperplane theory. For linearly separable cases we can find hyperplanes that classify objects into classes. SVMs can be use for this problem and some more cases. The two main improvements of SVM comes from these two ideas:
- soft margins: SVM tolerates a few objects to get misclassified and does a trade-off between maximal separation and minimizing classification error
- Using kernels for linearly non separable cases: Polynomial, Radial Basis Function, and others help you find more creative solutions.
Hi, this is from the lecture notes; please observe the use of the term “random plane”
this is a harbinger from the lecture notes,
Hello @ajeancharles
Thanks a lot for posting your question on Discourse. I will do my best to explain and give an answer to your quuestios.
Firstly, let’s revise some of the terms that we frequently use in machine learning and how they connect to one another.
Hyperplane: A hyperplane is a subspace whose dimension is one less than that of its ambient space. For example, in a 3-dimensional space, a hyperplane is a 2-dimensional plane. In different settings, hyperplanes may have different properties. For instance, a hyperplane of an n-dimensional affine space is a flat subset with dimension n − 1 and it separates the space into two half spaces.
Random Hyperplane: Random hyperplanes are used in various applications, such as dimension reduction and random projections. For example, in dimension reduction, random hyperplane tessellations can be used to partition a high-dimensional space into smaller regions This can be useful for clustering and classification tasks.
Random Plane: Random planes are a similar concept to random hyperplanes, but they are not necessarily subspaces. For example, a random plane can be defined as a set of points in a higher-dimensional space that satisfy a certain condition, such as being orthogonal to a randomly chosen vector. Random planes can be used in various algorithms, such as random projections for machine learning tasks.
Random planes can be used in random projections for SVMs, which is a technique used for dimensionality reduction. In SVMs, the goal is to find a hyperplane that maximally separates the different classes in the training data. A hyperplane is a subspace whose dimension is one less than that of its ambient space. The hyperplane is determined by finding the hyperplane that has the largest margin, which is defined as the distance between the hyperplane and the closest data points from each class. Once the hyperplane is determined, new data can be classified by determining on which side of the hyperplane it falls. In summary, while random planes are not directly related to SVMs, they can be used in random projections for SVMs, which is a technique used for dimensionality reduction.
I hope I was able to help you. Please feel free to ask further questions,
Regards,
Can
Thank you for the quick response. My question is an abstract one, I am sure you know that in both cases, you are classifying data by separating it by 1 plane (optimal plane) in the case of SVM and using many planes in the case of random planes. It feels like a generalization of the concepts; forget about the details.
What about random forests? can I find an isomorphism from each tree split to a plane split?
By the way, hyperplanes are also planes. And most planes are hyperplanes in the right context.