Hi All,
After go through the lessons, I am now confusing between instruction fine tuning, using prompt template, aren’t they also refer to prompt engineering?
If instruction fine tuning is not prompt engineering, then whats difference with soft prompt tuning (1 of the PEFT method)?
Appreciate for feedback.
Thanks,
Ming
Hi Ming,
Prompt engineering does not involve any training. Prompt engineering is simply the process of figuring out a prompt that is likely to yield the best result from the LLM. Again, there is NO training involved. In other words, the LLM’s weights are not changed. We simply spend time figuring out the most appropriate input to give to the LLM so that it generates the best response.
On the other hand, IFT involves actually training the model by changing its weights. In the process, we obtain a completely new, instruction fine-tuned model. The goal of instruction fine-tuning is (generally) to obtain a model which is capable of following instructions while the goal of prompt engineering is to obtain the best possible response from the LLM.
Coming to soft prompt tuning, instruction fine-tuning is different since in soft prompt tuning, the model is completely frozen and only the soft prompts that are added to the input embedding are learnt. In fine tuning, there are typically billions of parameters to train while in soft prompting, there are only a few thousand parameters to train.
Hope this helps!