# Text embeddings

**URL:** <https://community.deeplearning.ai/t/text-embeddings/569167>\
**Category:** Understanding and Applying Text Embeddings with...\
**Created:** [February 10, 2024, 8:55pm UTC](https://community.deeplearning.ai/t/text-embeddings/569167 "2024-02-10T20:55:28Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Manish\_Kamble](https://avatars.discourse-cdn.com/v4/letter/m/7feea3/32.png) [@Manish\_Kamble](https://community.deeplearning.ai/u/Manish_Kamble)\
**Post date:** [February 10, 2024, 8:55pm UTC](https://community.deeplearning.ai/t/text-embeddings/569167/1 "2024-02-10T20:55:28Z")

</div>

What are some other ways to create text embeddings?  
I am trying with LLM’s but chatgpt comes with added cost, huggingface models are two large and I dont have sufficient memory to run those models.

Is there any other way to create text embeddings for 100000 rows of review texts?

---

<div class="post-metadata">

**Author:** ![balaji.ambresh](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/balaji.ambresh/32/128697_2.png) [@balaji.ambresh](https://community.deeplearning.ai/u/balaji.ambresh)\
**Post date:** [February 11, 2024, 4:53am UTC](https://community.deeplearning.ai/t/text-embeddings/569167/2 "2024-02-11T04:53:56Z")

</div>

Have you seen this tutorial on [creating word embeddings](https://www.tensorflow.org/text/tutorials/word_embeddings) ?

BPE is effective since the number of OOV tokens is usually much lower than word level embeddings. You can always train a subword tokenizer and an embedding layer from scratch with a restriction on vocabulary size. Alternatively, you can warm start the embedding layer with pretrained weights for vocabulary in a custom subword tokenizer.

---

<div class="post-metadata">

**Author:** ![Manish\_Kamble](https://avatars.discourse-cdn.com/v4/letter/m/7feea3/32.png) [@Manish\_Kamble](https://community.deeplearning.ai/u/Manish_Kamble)\
**Post date:** [February 11, 2024, 12:51pm UTC](https://community.deeplearning.ai/t/text-embeddings/569167/3 "2024-02-11T12:51:52Z")

</div>

@balaji.ambresh  
Thanks for replying.

I will try this method, thanks!  
Is it effective for large number of text data though?

---

<div class="post-metadata">

**Author:** ![balaji.ambresh](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/balaji.ambresh/32/128697_2.png) [@balaji.ambresh](https://community.deeplearning.ai/u/balaji.ambresh)\
**Post date:** [February 11, 2024, 2:40pm UTC](https://community.deeplearning.ai/t/text-embeddings/569167/4 "2024-02-11T14:40:12Z")

</div>

Play with vocabulary size and embedding dimension to tackle the memory constraint on your system.

---

<div class="post-metadata">

**Author:** ![kwiseth](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/kwiseth/32/80632_2.png) [@kwiseth](https://community.deeplearning.ai/u/kwiseth)\
**Post date:** [March 29, 2024, 11:09pm UTC](https://community.deeplearning.ai/t/text-embeddings/569167/5 "2024-03-29T23:09:15Z")

</div>

BPE: [Byte code embedding](https://en.wikipedia.org/wiki/Byte_pair_encoding)   
OOV: [Out of vocabulary](https://kilthub.cmu.edu/articles/thesis/Learning_Out-of-Vocabulary_Words_in_Automatic_Speech_Recognition/21677660)