What exactly does "dense vector space" means in embedding layer?

I know what vector space is, it is set of vectors V over field F and is also represented as \mathbb{F}^n and has to follow certain axioms and are closed under binary operations \left\{ V, +, \cdot \right\}

But what does dense vector space mean here. This is blocking me to learn about layer working.

Please read this.

1 Like

After reading through the link that Balaji provided, I think that they are just using “dense” as a contrast to “sparse”. In a sparse representation, the output vectors tend to have lots of zero entries and just a few non-zero ones. The point is that the embedding vectors that are generated by the trained model tend not to have many zero elements.