Like in the picture, [1.2, 1.6] * [1,1] will get the point on the same direction as [1,1]
" To choose that orange vector note that the line y equals x is actually
the span of the vector with coordinates 1, 1.
So let’s use that vector 1, 1.
Now if you take the dot product of that first row in the table and the orange
vector, you’re essentially saying take 1 times the point x coordinate and
one time the point y coordinate to find the new location projected along the line."
I understand that two vector’s dot product is equal to l2-norm u * l2-norm v * cosin, if norm are on the same direction then cosin = 1, but I can’t relate this to two different direction vector… as the angle is not 0, so where is the cosin?
Sorry to go all math geek here, but what they show is that if you assume that the Law of Cosines is already proven, you can derive our desired formulation from that:
A \cdot B = |A||B|cos(\theta)
So what they’ve shown is that the two statements are equivalent, but they didn’t really prove either of them.
But it turns out it’s not that hard to prove the Law of Cosines by using the definitions of sin, cos and the Pythagorean Theorem from which you can also easily get the familiar identity:
Let me put it this way - when you take the dot product of a vector with a unit vector, you get the projection length of the vector along that unit vector. You want to prove this? Try, and I can provide a starting point:
Given the statement above, if you need the projection length of any vector on y = x, then ask youself: what’s the unit vector along y = x? If you get the right answer, you will find some square root of 2.
You want to know the projection of the vector (1.2, 1.6) on y = x? You take the dot product of this vector with the unit vector.
So I think I get the point by the geometric definition but not the algebraic definition(hope I don’t confused myself again).
Here is my thoughts, when we doing the projection(dot product), we are using norm x multiply cosin angle to get length e, then use length e multiply the norm y to get the scaled final points on the dotted line.
Since in this case we are only trying to get the length e, so after the dot product of two vector, we are dividing the norm y to get the length e.
But I am having a hard time now to understand why the two definitions of the dot product (algebraic vs geometric) are consistent as geo def is involving the cosin angle.
ChatGPT looks like is using algebraical rule to explain geo def, what I want to understand is how these two rule are the same without using algebra one to get the cosin angle:
Since you mentioned “dotted line”, I suppose you were referring to the graph above that I posted in my last message.
I suppose your “e” was my “l” and your “angle” my “theta”, because that’s what I got by multiplying the norm of vector x with cosine theta.
Now, the problem is, I didn’t use your “norm y”. My “l” is already the projection length, if projection length is your goal.
Your ChatGPT’s answer does not look like it is about projection, rather it is only about dot product. In this case, would you revise your prompt for ChatGPT to specifically ask about “finding projection using dot product” and try to re-understand the part that you don’t understand?
Note that, while we use a specific case of dot product to find projection, the general dot product rule does not always give us the projection length.
After that, would you please share your understanding again, so we can discuss it?
definition of projection to me is like to get the length e on Y by using cos rule x * cos theta
definition of dot product to me is like to get after projection on Y, get the value of X which is scaled by Y, so the new vector will be longer than vector Y.
So my original question was kind of resolved now, as
1+1 is the dot product of point [1,1] and [1,1] and then since we are not asking for scaling value, so we divide dot product by the norm [1,1] which is square root of 2
But while I was thinking about this process, a new question came to my mind, which is why the two definitions of the dot product (algebraic vs geometric) are consistent as geometric definition is involving the cos theta.
Like in my example why x1 * y1 + x2 * y2 can be written as square root of (x1 * x1 + x2 *x2) * square root of (y1 * y1 + y2 *y2) * cos theta?
It is, by definition, true that , and since “adjacent” is just the length of projection, you get that “x * cos theta”.
I will never define dot product with projection. Dot product is bigger than projection. Projection is just a specific use case of dot product when the vector y is an unit vector. By definition, unit vector is a vector that has a length of 1.
@Bio_J, please don’t start from projection and then try to define dot product. It’s like when someone uses a pencil to poke a hole on a piece of paper, that person then defines pencil as a tool for poking. Pencil is for writing, but it can poke a hole on a piece of thin paper. You can use dot product to calculate projection when the vector y is an unit vector, but this is only a very special use case because vector y is not always unit. When vector y is not unit, we are not calculating projection.
Even if you wanted to “break down” dot product into 2 steps:
take away norm y for a moment to first get the projection length
put the norm y back so the final answer is “e” “scaled” with norm y.
This is still an unusual way of understanding dot product.
Calling it a “new vector” is wrong. The result of a dot product is a scalar.
Saying it “longer” can be wrong, because you may be multiplying two numbers both less than one.
Was it the lecture that was asking you to keep thinking about “scaling”? You used this term for many times. I don’t understand it that way. This previous post summarized how I do. I don’t call anything scaling. The square root of 2 appears because it is in the representation of the unit vector along y=x. If we were not asking about the projection on y=x, but the projection on y=0, then there will be no square root of 2 because the unit vector along y=0 is just [1, 0].
If you ask for my recommendation, I would recommend you to think in the way I have presented in the previous post.
By the way, in my opinion, algebra and geometry are not two unrelated domains. Algebra represents geometric concepts in variables and equations. For dot product, that algebraic expression always includes cosine theta, and when you draw something out, you will include the theta as well. Theta is always there.
“Projection” is a very geometric idea, and I think we don’t have an algebraic understanding that can completely take geometry away. It seems to me you were trying to understand projection without any geometry idea, but this seems unreasonable to me.
It’s like Dot Product = Projection × Norm of Y in this case, I see what you mean here, so it should be Dot Product = Projection × unit vector.
Actually the lecture is not vey starter friendly on the dot product part…(as least I feel this way), dot product is the most important concept in the linear algebra PCA and the lecture didn’t explain much about why it worked that way.
I feel I circular thinking about this cos theta part,
it feels like a chicken egg question, I have to use this equation
to explain
But where is this equation
coming from? Why dot product of a and b equal to norm a * norm b * cos theta?
Yes! that is correct, [1,1] has the norm of square root of 2. I was thinking about the same but my definition of unit vector was bad so I didn’t think about word “unit vector”.
It would be [1/square root of 2,1/square root of 2 ] as the unit vector have equal 1, so norm of this vector is 1 on y=x.
This is exactly the way that I don’t prefer, because it’s like defining dot product as an operation that’s based upon the idea of projection. I agree that they are related but that’s not the relation I would agree. I would say that projection is a special use of dot product, or in other words, Projection = Dot Product when vector y is an unit vector (i.e. the norm of y is 1).
Yes. So my way might not be the same as the lecture, but if I want to calculate the projection of the tabulated vectors below on y=x, here is my step for the first vector:
It’s just the dot product of the vector with the unit vector along y=x, so no scaling whatsoever.
I am not the right person to comment on this because I am not mentoring this course and I have not completed it, but I believe Luis had a plan.
I learned dot product at high school and I do think the understanding that projection is a special use case of dot product is helpful. Dot-producting a vector with an unit vector to get the projection on the unit vector is also a very handy rule to remember.
I found a PCA lecture and did a few skips to find the following slide:
like why x1 y1 + x2 * y2 = |x| * |y| * cosin
, and since “adjacent” is just the length of projection, you get that “x * cos theta”.
