# About word embeddings in the CBOW model

**URL:** <https://community.deeplearning.ai/t/about-word-embeddings-in-the-cbow-model/233591>\
**Category:** NLP with Probabilistic Models\
**Tags:** week-module-4\
**Created:** [November 22, 2022, 10:39am UTC](https://community.deeplearning.ai/t/about-word-embeddings-in-the-cbow-model/233591 "2022-11-22T10:39:23Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![green\_sunset](https://avatars.discourse-cdn.com/v4/letter/g/e9c0ed/32.png) [@green\_sunset](https://community.deeplearning.ai/u/green_sunset)\
**Post date:** [November 22, 2022, 10:39am UTC](https://community.deeplearning.ai/t/about-word-embeddings-in-the-cbow-model/233591/1 "2022-11-22T10:39:23Z")

</div>

Hello everyone, In course 2 (week 4) of the NLP specialization, it is explained that word embeddings can consist of the columns of the matrices W1 or W2, or the average of the latter, in the neural network associated with CBOW. Before I learnt about CBOW, I was thinking that one would simply use the hidden layer representation of each input word (meaning that the one hot encoding of that word would be the input of the network) as the feature representation of that word. Would you think that this approach could work? Why is that possibility put aside in the CBOW model?.. Thank your for your comments and answers in advance 🙂  
Michel

---

<div class="post-metadata">

**Author:** ![arvyzukai](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/arvyzukai/32/165165_2.png) [@arvyzukai](https://community.deeplearning.ai/u/arvyzukai)\
**Post date:** [December 1, 2022, 7:12am UTC](https://community.deeplearning.ai/t/about-word-embeddings-in-the-cbow-model/233591/2 "2022-12-01T07:12:00Z")

</div>

Hi Michel,

I do not fully understand what are you proposing? To me, the way I understand your post it seems very similar to _Embedding_ layer ([docs](https://trax-ml.readthedocs.io/en/latest/trax.layers.html?highlight=embedding#trax.layers.core.Embedding) and [source code](https://github.com/google/trax/blob/2c13dff897256ca540652d51efce723e50beef0a/trax/layers/core.py#L130)). Here token/word\_id is mapped to a vector (if I understand you correctly as “the hidden layer representation”) of certain size. Is it the case or are you thinking about something different?