How do we obtain the embeddings from CBOW?

Max_Rivera · October 11, 2022, 3:27am

With a skipgram, the first weight matrix between the input and hidden layer is what we keep as the word embeddings. But CBOW is the opposite, with context words predicting target word. So which weights in the network are we keeping for the word embeddings?

Elemento · October 11, 2022, 7:32am

Hey @Max_Rivera,
The word embeddings in both the cases are actually obtained from the same weights matrix. The only difference resides in how the inputs, network and the outputs are structured. For more information about the same, you can refer to this detailed blog. Let me know if this helps.

Cheers,
Elemento

Topic		Replies	Views
About word embeddings in the CBOW model NLP with Probabilistic Models week-4	1	519	December 1, 2022
Intuition behind using the weights of a CBOW model as word embeddings NLP with Probabilistic Models week-4	2	569	September 5, 2023
How are word embedding calculated end to end NLP with Sequence Models week-1	6	599	January 10, 2023
How do we get the embedding matrix from the word2vec model? Sequence Models	3	716	October 10, 2022
CBOW context words to vector for creating embedding NLP with Probabilistic Models week-4	2	522	January 6, 2022

How do we obtain the embeddings from CBOW?

Related topics