Questions on how the projection works

Hi team,
in the video Dimensionality Reduction and Projection
I don’t get it why two vector multiply together will get the direction on the same line?


Like in the picture, [1.2, 1.6] * [1,1] will get the point on the same direction as [1,1]

" To choose that orange vector note that the line y equals x is actually

the span of the vector with coordinates 1, 1.

So let’s use that vector 1, 1.

Now if you take the dot product of that first row in the table and the orange

vector, you’re essentially saying take 1 times the point x coordinate and

one time the point y coordinate to find the new location projected along the line."

I understand that two vector’s dot product is equal to l2-norm u * l2-norm v * cosin, if norm are on the same direction then cosin = 1, but I can’t relate this to two different direction vector… as the angle is not 0, so where is the cosin?

1 Like

This paper seems to be a nice proof:

3 Likes

Sorry to go all math geek here, but what they show is that if you assume that the Law of Cosines is already proven, you can derive our desired formulation from that:

A \cdot B = |A||B|cos(\theta)

So what they’ve shown is that the two statements are equivalent, but they didn’t really prove either of them.

But it turns out it’s not that hard to prove the Law of Cosines by using the definitions of sin, cos and the Pythagorean Theorem from which you can also easily get the familiar identity:

sin^2\theta + cos^2\theta = 1

well, after a year I am back to the same point…… so why we need to divide root of 2 here

Hello @Bio_J,

Let me put it this way - when you take the dot product of a vector with a unit vector, you get the projection length of the vector along that unit vector. You want to prove this? Try, and I can provide a starting point:

Click here for the starting point

Given the statement above, if you need the projection length of any vector on y = x, then ask youself: what’s the unit vector along y = x? If you get the right answer, you will find some square root of 2.

You want to know the projection of the vector (1.2, 1.6) on y = x? You take the dot product of this vector with the unit vector.

Cheers,
Raymond

2 Likes

Thank you Raymond!

So I think I get the point by the geometric definition but not the algebraic definition(hope I don’t confused myself again).

Here is my thoughts, when we doing the projection(dot product), we are using norm x multiply cosin angle to get length e, then use length e multiply the norm y to get the scaled final points on the dotted line.

Since in this case we are only trying to get the length e, so after the dot product of two vector, we are dividing the norm y to get the length e.

But I am having a hard time now to understand why the two definitions of the dot product (algebraic vs geometric) are consistent as geo def is involving the cosin angle.

ChatGPT looks like is using algebraical rule to explain geo def, what I want to understand is how these two rule are the same without using algebra one to get the cosin angle:

imagelike why x1 y1 + x2 * y2 = |x| * |y| * cosin

Thank you!

Hello @Bio_J,

Since you mentioned “dotted line”, I suppose you were referring to the graph above that I posted in my last message.

I suppose your “e” was my “l” and your “angle” my “theta”, because that’s what I got by multiplying the norm of vector x with cosine theta.

Now, the problem is, I didn’t use your “norm y”. My “l” is already the projection length, if projection length is your goal.

Your ChatGPT’s answer does not look like it is about projection, rather it is only about dot product. In this case, would you revise your prompt for ChatGPT to specifically ask about “finding projection using dot product” and try to re-understand the part that you don’t understand?

Note that, while we use a specific case of dot product to find projection, the general dot product rule does not always give us the projection length.

After that, would you please share your understanding again, so we can discuss it?

Cheers,
Raymond

Hi Raymond,

Sorry for the confusion, I am weak on the definition of the projection and dot product.

Here is what I mean.

definition of projection to me is like to get the length e on Y by using cos rule x * cos theta

definition of dot product to me is like to get after projection on Y, get the value of X which is scaled by Y, so the new vector will be longer than vector Y.

So my original question was kind of resolved now, as

1+1 is the dot product of point [1,1] and [1,1] and then since we are not asking for scaling value, so we divide dot product by the norm [1,1] which is square root of 2

But while I was thinking about this process, a new question came to my mind, which is why the two definitions of the dot product (algebraic vs geometric) are consistent as geometric definition is involving the cos theta.

Like in my example why x1 * y1 + x2 * y2 can be written as square root of (x1 * x1 + x2 *x2) * square root of (y1 * y1 + y2 *y2) * cos theta?

thanks

Your concept of the dot product is not correct. What you’re thinking of is more of a scaled vector addition.

Can you also explain what is the relationship between projection and dot product then? if my understanding of dot product is incorrect