In addition to @saifkhanengr‘s great reply:
Transfer Learning works well:
- in general if you have (implicit) representative features learned in the pre-trained model which help in the fine-tuning
- e.g. if you learn / transform a certain style, see also: transfer of style
In the end a deep learning model can learn what is contained in the data. If the data is sufficiently representative, compared to the new specialised tasks, transfer learning should be worth a look! Then you can leverage the pre-trained model (which is usually also high efforts, super expensive and difficult to create in the first place).
see also this thread.
Best regards
Christian