C2_W3_Transfer Learning

Course Q&A Machine Learning Specialization Advanced Learning Algorithms
1

Hello, it was mentioned in the lecture that the data used for your own application and data used for pretrained model should be the same to make things work for training data.

I would like to ask how “same” the data should be. For example, to train a digit classifying NN, is it sufficient for the pretrained model to be trained on an image, or do they have to have a same size? I’m asking this because I was thinking about what if I have a different input dimension (especially for things like audio or text as we might extract different features)? How should I proceed with transfer learning?

Thanks.

2 Likes
2

All the images should be the same size.

1 Like
3

Thanks for reply! However, if that’s a condition, wouldn’t it be too hard to used a pretrained model trained by someone else? It is quite natural that your dataset would have a different input dimension.
Or can be try adding a simple layer in front such that our input will match the input to the pretrained data after going through the layer?

Thanks!

2 Likes
4

You would need to pre-process (i.e. resize) the images before you can use them.

4 Likes
5

A similar distribution of data is necessary for transfer learning. Resize photos to fit the input scale of the pre-trained model. Make sure that the data structure for text or audio matches the expectations of the model. Adjust the model to make it more responsive to particular features using your dataset. For best results, track performance and modify the parameters as necessary.

3 Likes
6

Hello @jaejun02,

We are multiplying matrices in neural network, and if the shape does not match, the multiplication won’t work. In principle, you may add an additional layer in the front to bridge the shapes and see how it goes, but for image, it is very common and is easier to just resize the image to the right size and the result usually isn’t bad.

Cheers,
Raymond

1 Like
7

Hi @jaejun02 this is a great question! I had the same question when I wanted to start using pre-trained models, there are several approach that you can take for this. It is true you need to pre-process your data to have the same size, that’s why some pre-trained models are better for some task than others, you can customize some pre-trained models to use your own data, it might be a more difficult approach but it is possible.

I hope this helps!

1 Like