How to gain intuition for finetuning LLMs

AI Discussions
1

Once you know the theory of machine learning, transformers, and all the extras e.g. LoRA and QLoRA, and you know how to use the Hugging Face interface, you essentially know what you need to start finetuning, but that doesn’t mean you’re any good at it.

There’s so many parameters and having studied the theory, you know what each one means theoretically, but practically you don’t have enough experience to be able to make insights like “hmm yeah it seems the learning rate is too high”. Aside from just practice how do you gain these intuitions?

As an example, I have a notebook here and I feel like I set up everything right, but as you can see, the evaluation before and after is unchanged.

2

The first step to have better intuition to under the dataset you are using, your score shows mismatch and match score to be almost equal.

so kindly give some details about what kind of dataset you are using, how you are splitting the dataset, how are defining the columns.

Regards
DP

1 Like
3

Thank you very much for your answer. The dataset is this one openlifescienceai/medmcqa · Datasets at Hugging Face
I tried training on a large number of samples, varied the learning rate, effective batch size, and LoRA parameters but no impact at all from that. My guess is that LoRA isnt powerful enough to teach new information?