Transfer learning: Can you have different output data (y)?

[FerDDei]

Hi there,

I’m interested in how transfer learning can help models with little data perform better by pre-training with a task that has similar low-level features and a lot more data.
You said that task A and task B should have the same input data X, but what about the output data y?
For example, suppose you pre-train a model for image recognition, where the output is a binary classification, and then fine-tune it for radiology diagrams, where the output is a multi-classifier for different conditions detected in the diagrams. (I’m not a radiologist, so please excuse any radiology inaccuracies!)
In this case, the training data would have 10 outputs of different diagnoses, which is different from the single output from the image classification.
My guess is that the radiology model would still derive benefit from pre-training, because the low-level features that are pre-trained from the image recognition would be useful for both a binary and a multi-class classifier, such as edge detection. But I also wonder if there would be more benefit from going from binary to binary than from binary to multi-class.

I would love to hear your thoughts on this.

Thanks!

[gent.spah]

Yeah you could use the binary model as pretrained, dropping some final layers adding an output, maybe adding other prior layers too and fine tuning it. You can see such applications in the TF Developer (as far as I remember) and TF Advanced Techniques specializations.

Is it better to use binary to binary or binary to multiply outputs?, it depends which features are you using from the pretrained model, what nature are the data in each cases, how long will further tune etc. In general its a not a straightforward matter to clarify and if you fine tune long enough might not make much difference.