What is R-matrix?

Course Q&A Natural Language Processing NLP with Classification and Vector Spaces
1

As I understood it is a Matrix where each row is corresponding to each row in X data. After dot X and R some transformations appear and we have transformed X data. So, in few cases like example in your video we don’t get ready result from this multiplication? And only intermidiary result with which we should do something to get end result, like PCA? For example to compare with Y.

So, can you exaplain me more about R matrix, please? I a bit not fully understood sence of it. Or is it ok, that it is something abstract and I shouldn’t have clear understanding ?

2

Hi @someone555777

I think essential to understand linear transformations if you want to understand Deep Learning. It’s not necessary for making use of of DL but it’s essential for understanding.

I would suggest this video - Linear transformation examples: Rotations in R2 to understand the rotation in two dimensions (and maybe this in three dimensions).
To be fear, his whole crash course on Linear Algebra I think is very useful for understanding DL under the hood.

Cheers

3

thank you for links, but I don’t understand an answer on my question.

So, multiplication of english words vectors matrix (E) by R matrix, that contain coefficient for each element in E to transform it to near by sense, gives us new “vectors of senses”, right? And we try to get all synonims for each of this vectors in matrix with all french words vectors (F)? Do we need this, because senses of words in different languages can rarely have absolutely the same meaning? Specially if count different contexts?

4

That is true and they do not. The best thing you can do is approximate and to achieve your desired goal (for example, an accuracy that is good enough).
You want to learn these weights so that after multiplication you get new “vectors of senses” as close as possible.

5

A bit not understand what did I say incorrect? I don’t see that your answer contradict to my answer.

6

Nothing, you said everything correct, I just elaborated.
There is no perfect translation (just approximate) in real life too.
Every word even in the same language might mean something different to different people.
So you should not expect the R matrix to magically “rotate” English word to French without any ambiguity :slight_smile:

1 Like
7

of course not, we trained model for this as I understand to find most appropriate weights in R-matrix.

But did I understood correct the whole princples of what you tried to explain in course in my answer?

8

There is some language barrier so I’m not sure, but I think you get most of it correct.
For example,

I’m not sure what you mean by that. Technically, transformation of English word embedding to French is mapping from English embedding space to French embedding space with the help of R matrix. So, for example to be concrete, if embedding for word “cat” is [1, -2, 0.3], the R matrix transforms it to [0.5, -0.2, 0.25].

And to elaborate on your other post in different thread:
Most probably there is no word in this exact spot, so you need to search somewhere around here. You calculate the hash for it and search only in the local region.

9

yes, that’s what I mean, thanks

10

and just want to clearlify, why is it named R-matrix and not just weights matrix as usually?

11

I’m not a big fan of naming things :slight_smile: but I guess I can try to explain my understanding:

  • R-matrix it is named that way because it is a Rotation matrix.
  • The concept of rotation matrices has a long history in mathematics, dating back to Euler’s work in the 18th century and earlier. The rotation matrix is always square - meaning it does not change the dimensionality of the output (while the “weight” matrix or “transformation” matrix can change the dimensionality of the output, in other words, rotation matrix is a subset of “weight” matrix).

Cheers

1 Like