Why are theta_i and e_j symmetric in GloVe model?

Course Q&A Deep Learning Specialization Sequence Models
1
  1. What does “symmetric” mean? Meaning after adequate training these two will be equal?
  2. If so, why will they be equal?
2

Hi Zhihan,

From my point of view, it depends on how we define the target and context words. The symmetric relationship holds when we define the context and target through close proximity (E.G., within ± 10 words of each other), which will result in X_{ij}=X_{ji}, which shows the frequency of word i and j appears close to each other. The loss function for the GloVe algorithm is to minimize the difference between \theta_i^Te_j with logX_{ij}, and I think that here \theta_i and e_j play symmetric roles due to the symmetry of X.

Thanks,
Kezhen