How do you approach the use of embeddings and LLMs in non-english languages (e.g. French)?

AI Discussions
, ,
1
  1. Not a great choice of base LLMs for fine-tuning in French (any decent equivalent of (monolingual) T5-base?
  2. Multilingual LLMs come with fat (mT5-base = 2.3GB, T5-base=0.9GB)
  3. What about embeddings, especially for vector databases (OpenAI works better than camemBERT in French, but API latency makes it slow for large number of documents)?