Is there any requirements for the data of the pre-training step during transfer learning (except the same type of data for both steps)?
Professor says about diversity (Video ’ Transfer learning: using data from a different task’ 7:45: ‘by training a neural network to detect things as diverse as cats, dogs, cars and people, you’re helping it to learn to detect these pretty generic features of images and finding edges, corners, curves, basic shapes.’)
In other words could be the transfer successful for face recognition task if the model was pre-trained for the handwritten digits recognition task.
Maybe. It depends on what the similarities are between the data you used for training and what your ultimate project is.
The more the training data resembles your data of interest, the better your results will be (because you could probably use a simpler model, or train it faster, or have a smaller data set, etc.).
Usually you want something that use a similar dataset, but is not limited. People test different things and try different approaches and evaluate what works.
This is the repository in which you can find some pre-trained models:
So what you want to do is to take a model previously trained with millions of observations that is able to identify patterns in data, and then you fine tune on your own dataset, so even though the model wasn’t trained with similar dataset you can still use it.
In this notebook I classified skin lesion using a previously trained model called Xception Which wasn’t trained with similar dataset, but I optimize the model with my own dataset to make it work better.