Week 2 - Question about single-task fine-tuning

[yingele]

In the lecture about fine-tuning on a single task it is said that the model which is fine-tuned on a single task may perform badly on other tasked compared to before the single-task fine-tuning.

Question - what is this “before”? Was this model fine-tuned before as for example described in the previous lecture, e.g., full fine-tuning on many different tasks? Or we are talking relative to the pre-trained model (not fine-tuned) and we are talking about degradation of tasks which the pre-trained model did well without any fine-tuning?

(couldn’t tag this with week2 as there’s no such tag)

[gent.spah]

@yingele

Or we are talking relative to the pre-trained model (not fine-tuned) and we are talking about degradation of tasks which the pre-trained model did well without any fine-tuning?

This is correct!

[yingele]

@yingele

Or we are talking relative to the pre-trained model (not fine-tuned) and we are talking about degradation of tasks which the pre-trained model did well without any fine-tuning?

Thanks!