Hi, everyone, I do have a question concerning the GloVe lecture.
In week 2, for the GloVe lecture. At the beginning of the lecture and even at the clarification before the lecture, Xij was said to be the number of times that j occurred in the context of i. (2:19)
To simplify this, using previous models like skip-gram, i is basically c(content) and j is t(target).
However, at time step (3:05), theta is expressed in form of i, and later i was labeled as the target, and not content as earlier described. This also affected j, and was treated as the content, instead of using it as a target.
I want to know if this is an error or just due to the fact that Xij is symmetric to Xji.
The fact is that i and j are just indices. There is no fixed convention that i denotes the target and j denotes the context and vice-versa. As a result, you can see Prof Andrew almost always described what i and j denotes whenever he used those indices, since they can be used interchangeably.
As for this, let me quote Prof Andrew only
And depending on the definition of context and target words, you might have that X_ij equals X_ji. And in fact, if you’re defining context and target in terms of whether or not they appear within plus minus 10 words of each other, then it would be a symmetric relationship. Although, if your choice of context was that, the context is always the word immediately before the target word, then X_ij and X_ji may not be symmetric like this. But for the purposes of the GloVe algorithm, we can define context and target as whether or not the two words appear in close proximity, say within plus or minus 10 words of each other. So, X_ij is a count that captures how often do words i and j appear with each other, or close to each other.