Range of cosine similarity: between 0 and 1

Course Q&A Natural Language Processing NLP with Classification and Vector Spaces
1

Lecture says,

image
image1315Ă—445 35.5 KB

What is to stop from getting a range between 1 and -1?

1 Like
2

Nothing stops it from getting a range between 1 and -1. In fact, cosine distance has all that range.

As for why the summary indicates that the value goes between 0 and 1? I can venture an answer:

When the C.S. = 1 the vectors are identical. This is one of the limits
When the C.S. = 0 the vectors are orthogonal (no match). This is like one of the limits
When the C.S. = -1 they are pointing in opposite directions - I would argue that in this case the similarity is out of the question.

It’s a weak answer but the only I can think of.

In summary, between 0 and 1 we are in the range of similarities.

Again, this is me venturing an answer. Lets hope for a better answer from someone more knowledgable.

3

Thank you! Something to think about.

4

Definitively! You got me thinking right here! Hopefully someone jumps in with more information!

1 Like
5

Hello @ajeancharles,

are the vectors’ components constrained to non-negative values only?

Raymond

6

If we are talking about embeddings, the vectors are not constrained to non-negative values only. So it is technically possible to have -1 as the cosine similarity.

Now, in Word2Vec, GloVe and other similar word embedding, it is rare to have these cases, and may be that’s why it is said that the range is 0-1.

7

Yes… but I think we will need @ajeancharles to investigate into the question and provide the context that can explain the range.

Non-negative components is one possible reason.

A term frequency vector contains only non-negative components.

A rescaled cosine similarity can also be a possibility.

Raymond

8

Thanks! I have to think about this.

9

Hey @ajeancharles,

It is in fact the intended range of cosine similarity. However, you will find that for the example discussed in the lecture video, the vectors represent the frequencies of 2 distinct words, i.e., the vectors have non-negative components only, as @rmwkwok pointed it out. Therefore, Younes stated that the cosine similarity ranges between 0 and 1. Let me quote an excerpt from the video for your reference:

Remember that’s in this example, you have a vector space where the representations of the corpora is given by the number of occurrences of the words disease and eggs.

Also, when Younes stated the range of cosine similarity, he ensured to state the following:

for the vector spaces you’ve seen so far, the cosine similarity takes values between 0 and 1.

Nonetheless, I will raise an issue regarding this, to perhaps add some pop-up, indicating that the actual range of cosine similarity is [-1, 1]. Thanks a lot for creating this thread.

Cheers,
Elemento

10

Hi @Elemento,

My two cents.

I think @Juan_Olano’s examples very well illustrate that, ultimately, if we want to remember one range, -1 to 1 should be the one to remember.

I think it’s worthwhile to first explain that the non-negativity is a special case because we construct our word vectors on term frequency, and it is such special case that the range is limited to 0 to 1 because the angle between any two vectors is always between 0 to 90 degrees. Graphically , the vectors are confined to the first quadrant, and so the angles are always less than or equal to 90 degree. Then, as we progress to learn word vectors with neural networks (when?), our word vectors’ components can take up any value, which makes it possible for the angle to be between 0 to 180 degrees. Graphically, our vectors can point in any direction, making the angle between two vectors to be able to exceed 90 degrees and comes the negative cosine similarity.

In this way it should make it a smoother transition from the lecture’s special case to the more general case?

I am not sure if the version of my course is still up-to-date, however, there is a reading item after the video for Cosine Similarity. It might be a good place for a longer explaination. That reading item actually re-emphasized the non-negativity and the range of the angle, therefore it might be good to expand from there?

Cheers,
Raymond

11

Thanks, I will do some reading.

12

Hi @Juan_Olano

I think you have typo here - 0 → 1.

Just for future readers.

Cheers

1 Like
13

Thanks! Fixed! Appreciate the heads up.

1 Like
14

The pleasure is all mine!

15

Hey @rmwkwok,

Although the reading item don’t emphasize on the negativity of the vectors yet, but that’s indeed a good suggestion. Let me pass on your suggestion to Mubsi.

Cheers,
Elemento

1 Like