Some confusion on Word2Vec model

[arvyzukai]

Hi @SurajKP79

There are some misconceptions in your post and your English is hard for me to understand. So let me address this point first:

@SurajKP79

Now we initialize the softmax parameter theta randomly and then …

That is not true. Softmax is just an operation that fits the vector to values from 0 to 1. An example.
In other words, the outputs (values) of the network are all over the place - negative, positive, big numbers, small numbers - and if you want them to be interpreted as probabilities (going from 0 to 1, and the sum of them to be equal 1), then you can use softmax.
So the theta is actually not a parameter (of the model), but the outputs of the model.

@SurajKP79

Is there values of the embedded matrix are randomly initialize and during training, it got a specific value? Hence after running the model, we can get our embedded matrix having word embeddings ( is it TRUE? )
if YES then how it is updated?

Yes the embedding matrix (values) are randomly initialized at the start of the training. These values then are constantly updated according how well the model predicts the targets - values that contributed to lowering the probability of the correct word are reduced and values that contributed to increasing the probability of the correct word are increased.

Cheers