PEFT/LoRA used for domain addaptation

I am interested if we can used PEFT/LORA methods to adapt given model to new domain. The main motivation is to avoid full-pre training cost for domain adaptation (like BloombergGPT).

Can somebody share thoughts about it. I will be grateful :slight_smile:

Challenges which I see:

a) We will need to use embedding mapping and tokenization used in original model but there may be domain specific words which are not represented in it.

Is there a way to “froze” the embeddings and tokens of words on which previous model was trained and add new ones which are domain specific? The new words will be mapped into existing space similarly like in soft promoting as I imagine it.

Excellent question, and the first thing to stumble upon when working in a specific industry, like Bio-medical, manufacturing. I have a similar query above. The HuggingFace lib has made fine-tuning of the tokenizer and the model (one might only fine-tune the model, as that works best in many cases), quite simple. But I am not sure if the latest openai model is available in HuggingFace repo , I see only openai-gpt, but I dnt think its the latest. So in such scenario how do we fine-tune the model. Secondly, as rightly queries, how do we update the tokenizer with peft / Lora. Will be good if someone can explain. We have done this with SLMs (Small language models!) like BART, few yrs back, but not attempted this with LLMs. Thanks.

1 Like