Question about Hot-encoding and the size of the corpus

In the lessons we saw /discussed the case of a 10K words corpus, so the vectors of a hot-encoded sentence would have a shape of 10K.

Is there any special consideration or a different strategy when the corpus is much larger, say 100K? or the same hot-encoding strategy applies, hence each word of a sentence would be represented by a vector with shape [100K,]?

Hi, @Juan_Olano !

We dealing with specially large datasets, one-hot encoding may not be the best approach as it uses way more memory than, say, integer encoding. Check sparse categorical cross entropy as it is the common loss function for this.

Thank you! Yes, that makes sense. I will investigate this option.