Does SFT and PEFT come under instruction fine-tuning categories? What is the difference between SFT and PEFT? In SFT are all the parameters updated i.e., it is called full-finetuning?
Hello Learner, SFT is indeed a form of instruction fine-tuning. As it involves training a pre-trained language model on a dataset of instructions and their corresponding desired outputs.Typically, all parameters of the model are updated during this process, which is why it’s often referred to as “full fine-tuning”.
Whereas PEFT (Parameter-Efficient Fine-Tuning) is not necessarily comes under instruction fine tuning technique as it is not a specific fine-tuning technique, but rather a category of methods designed to fine-tune llms more efficiently. It can be used for instruction fine-tuning, but it’s not limited to that use case. PEFT methods aim to adapt models using fewer computational resources and less memory. They typically update only a small subset of the model’s parameters or introduce a small number of new parameters. Some of the techniques involved in PEFT are LoRA, Prompt tuning etc.
1 Like