NLP - C1W4 - Multiple planes and point side locations

Course Q&A Natural Language Processing NLP with Classification and Vector Spaces
1

Hi, so this is really a linear algebra question-- But stated how it is provided in the lecture, every point will have a positive/negative dot product based the side of the plane upon which it is compared.

*I would also just tack on here-- My understanding, at least, to start with is that all vectors are defined vis a vis the origin of the plane in question. I mean you can move to other points from there, but that is where they all begin.

How then, exactly, does a plane get defined ?

image
image866×479 32.9 KB

However, from the ‘intuitive’ standpoint – Seeing as we are not actually given any coordinates for the planes here, you could kind of say the ‘positive’ side always lies ‘to the left’ side of the plane-- However, that also completely depends on which direction you are drawing the plane from (i.e. which side is left/right)–

Just wondering how does this work out in practice ? Also, is there some ‘general rule’ such that, simply looking at the provided diagram, you’d know which side is ‘left’ right ?

1 Like
2

hi @Nevermnd

that’s a good question.

The first thing to note is that the vector that defines the plane does not mark the boundary between the two sides of the plane. It marks the direction in which you find the ‘positive’ side of the plane. Not intuitive at all!

If we want to plot the separation plane, we need to plot a line that is perpendicular to our vector P.

I suppose the key point here is to get single hash values using multiple planes.

the side of the plane is determined by np dot product explained in the previous slide of the same week which explains how the hash value are assigned.

Regards
DP

3

@Deepti_Prasad thanks for the reply. I completed this course kind of ‘by accident’ this morning, or unlike DLS, I didn’t realize the labs here are all that is required-- Otherwise my work ethic has been to kind of complete the labs, think about those for awhile, and then go back and look at the quizes.

Obviously there were enough suggestions and I think I am decent enough at programming that I can just ‘follow the rules’, but say where random planes are used in LSH, I get the overall concept-- but perhaps not the overall math that defines a plane as a plane (I mean you can do a dot product of two vectors two, right ? That doesn’t magically establish them as a plane).

I’ll have to think about and return to this question.

1 Like
4

@Nevermnd I really didn’t get your response, my response was totally addressing your doubt concerning on how one choose the positive and negative side of a plane, i.e it is related to the vector point.

vectors don’t have starting or ending positions, just directions. So take the vectors <1,0,0> and <0,1,0>. These vectors will define a plane that only goes in the x-y direction, but the problem is, they will work for any z-coordinate. So you need some starting point to anchor your plane.

Also to response to your second comment where you are stating about two vector points,
If two planes are perpendicular, their normal vectors will also be perpendicular.
If a plane contains two points, it contains the line through those points; in particular its normal vector is perpendicular to the direction vector of the line.

I hope m not confusing you :blush:

algebra is tricky only until you can image the question in some shape or form.

Regards
DP

5

Let me get in more in depth when I have some time to sit down with Photoshop.

I mean grant me the image cited above-- It is reasonable to think (or suggest), say, at least an X,Y,Z ‘plane’, at least in the conventional sense of the term, and then it gets too hard to visualize, mentally, as we enter hyperplanes, but the whole concept is they all share the same origin.

Afterall, that is how you define a vector, isn’t it ?

Again, even in the simple 2D image above, please explain to me what ‘origin’ these planes share ?

There seems to be none, just random criss-cross lines.

1 Like
6

A hyperplane is simply a flat (linear) surface of one less dimension than the space in which it is embedded. So in 2D space, a hyperplane is a line, which is what is happening in your diagram. A line does not have to go through the origin, right? You can pick any point and draw a line through it at any angle. So you characterize it by a point and a normal vector. The direction of the normal vector is what I think Deepti’s point is about the positive side of the plane (line in this example).

1 Like
7

Thanks Paul… Like I said… Why I think this is a Linear Algebra question… Obviously I ‘get’ what a ‘line’ is, but I thought we were only allowed to have vectors.

Or let’s play a little ‘devils advocate’, lets say I have a point and a norm-- How do I magically have the mx+b of that line that is now no longer in relation the origin (directly) ?

Or I don’t see how just a point and a norm informs me about everything else that lies on that plane (one side or another-- Without a formula). I am missing it.

1 Like
8

@Nevermind

Iam pretty sure you know vector is a combination of magnitude and direction from a given point? right!!!

Say I am going to a destination and I want to know which direction I need to take to reach the destination which is 500 meters away from my given point. The response I get is to go south. So the vector point here would be south direction (direction) 500 meter (magnitude)away is my destination, and this vector point is not constant, it can change in any direction based my destination and my current position.

So usually when I am at point, the usual way is to form x to y axis, similarly in space vectors can be present in multiple planes (in any direction), so as stated in my previous comment

If two planes are perpendicular, their normal vectors will also be perpendicular.

If a plane contains two points, it contains the line (this line is created through those vector points based on the direction vector ) through those points; in particular its normal vector is perpendicular to the direction vector of the line.

vector is always a point but when it follows a particular direction, its coordinates are defined with the help of x or y axis, or you could say z axis based on dimensionality of vector points in a given space.

feel free to ask if you didn’t understand!

9

I hope you are not disputing the fact that the line is uniquely determined by the normal vector and a point on the line. If you accept that, then:

It’s not magic, you just have to solve. If a line has slope m, then a line that is perpendicular to that has slope -\frac {1}{m}. Proving this requires knowing a bit of trigonometry, in particular:

sin(a + b) = sin(a)cos(b) + cos(a)sin(b)
cos(a + b) = cos(a)cos(b) - sin(a)sin(b)

Apply that with b = \frac {\pi}{2}. Left as an exercise to the reader. So if the normal vector is (x,y), the slope of that is \frac {y}{x} (rise over the run, right?). From that we can compute m for the line. Now I have y = mx + b and I have a point on the line, so it’s easy to solve for b.

Of course this is only the 2D case, but it generalizes to higher dimensions.

1 Like
10

You might find this discussion a little bit helpful too: Question on locality-sensitive hashing

3 Likes
11

Thanks @jyadav202, Paul.

Sorry for the delay… I’ve been dealing with some unexpected health issues lately and just got out of the hospital again…

So, first off, the degree of Linear Algebra knowledge I have is dealing directly with vector/matrix operations, which I realize is really technically no different when such things are graphed-- Yet early on, when I decided Physics was not my favorite subject, I kind of consigned that kind of thinking to ‘okay, I’m just going to ignore this’ part of my brain-- Thus sorry if a few of my questions are a bit basic.

I tried starting out by putting together a conceptual counter example:

image

Or, granted we are limiting ourselves here to only a 2D plane, and the stipulation a norm from any plane must be in reference to the origin to start with, then certainly some planes you can get to, but you need some combination of vectors to do so…

Then, as I worked on this, I realized-- wait a minute. I am probably wrong. I think you can reach any plane in the same number of dimensions requiring only one vector from the origin to do so.

That vector may not run through the quadrant you wish it to, but you can still get there with only one vector-- Is this ‘re-thinking’ more correct ? And if so, beyond just the 2D case, does this fully generalize ?

The second part of my question Paul (and no I am not disputing what you suggest as the relationship between a norm and a vector), is assuming say we have this vector:

image
image602×530 6.48 KB

In Python I can just then go say:

>>> my_vector = np.array([1, 1.4])
>>> my_norm = (np.linalg.norm(my_vector))
>>> print(my_norm)
1.7204650534085253

And, okay ! Great… But now I have… a scalar (or not, at least up front obviously, another vector)… So what exactly is this scalar and how am I supposed to interpret it ?

1 Like
12

Normal vectors don’t need to be from the origin. The normal (pun intended) way to do it is that they’re from a point on the line itself.

On your last question, the norm of the normal vector is not relevant. You could use it, but it just makes things more complicated than need be. The easy way is what I briefly described in my previous post.

All you need is the slope of the normal vector, which is:

m_n = \frac {1.4}{1} = 1.4

So as I showed above that means the slope of the line itself is:

m_l = -\frac {1}{1.4} = -0.71428....

Then we have a point on the line, which is (1, 1.4) so we just substitute that and solve for b in y = mx + b:

1.4 = -0.71428.... * 1 + b
b = 1.4 + 0.71428.... = 2.11428....

1 Like
13

Ohhh… okay, I think I am starting to get it… I mean to be honest, even so far back as grade school I didn’t realize the slope of a norm or perpendicular line was so simple, or it was never seen as important enough to know about. It makes sense though-- you are basically just inverting it.

Still though, so what then is the actual scalar norm (in this case ~1.720)… supposed to be telling us ? I mean it can’t be a point with only one coordinate… And it is not the slope… (Obviously we are calculating a lot of norms in this application…)

1 Like
14

I don’t remember exactly when I learned that theorem, but it was probably analytic geometry in the 10th or 11th grade. I don’t think they made that big a deal about it.

It’s easy to prove if you just draw the diagram and then use the trig identities that I gave above:

sin(a + b) = sin(a)cos(b) + cos(a)sin(b)
cos(a + b) = cos(a)cos(b) - sin(a)sin(b)

Use those formulas with b = \frac {\pi}{2}.

The slope of one line is tan(a) and the slope of the other one is tan(a + \frac {\pi}{2}). Apply those formulas and you’re done.

1 Like
15

The norm of the normal vector tells you nothing of relevance in the usual case in which the normal vector is based at a point on the line. In fact, there are an infinite number of vectors that could be used as the normal vector: multiply by any positive scalar constant and the vector is still normal to the line. All you care about is the slope of the normal vector. From that you can compute the slope of the line itself. Now you’ve got the slope of the line and a point on the line and you solve for b as I showed above.

1 Like
16

Thank you very much, Jadav! Will look into it.
My best regards,
Vasyl.

1 Like