I’ve already learned that, if my final target NN is Z, and well-trained NN A is strongly related with target Z, then I just freeze most of the layers in A, and only train the last few layers in A for Z. That’s how we inherit the A’s “knowledge”, this is one to one (A transfer to Z) transfer learning.
Now, my question is that,
almost everything is the same, but I want to inherit or transfer 2 well-trained NN (lets say A, B) for Z. What should I do about it ? In other words, this transfer learning is not one to one, but many to one (A,B to Z) now. Any master could give me some hint or some key words I could google about it at least ?
Its a tricky proccess I am not sure it can be done! In the MLops specialization they introduce some transfer learning techniques like Teacher-Student models, I think its course 4, go and and have a look at it.
But to infuse many networks combined to one I dont know if it can be done because how would you merge the many different architectures together and their different parameters.
That’s the reason I am asking, I already done that class, that’s how I know this one to one transfer learning.
I believe there must be a way to do this. Maybe not the same way like before which only train the last few layers, but whatever new strategic to do this.
May I ask, if I have well trained NN A and B, now, I just want to “combine” then together to become my new NN Z (Z = A “+” B), is there a way to do that?
Maybe I can rephase and split my question into 2 stages.
Stage 1: for example, if the NN A can classify cat and dog very well. Another NN B can do classification between human male and female very well. A, and B are well trained, the NN structure are even the same.
Now, my new mission is to do the classification among cat, dog, male, female. So, is there a way I can do it through A, and B ? that’s what I meant, is there a way to “combine” A and B? or what are the strategies to do this ? Of course, I am not talking about retraining everything. Let’s assume, those NN A & B, I just download somewhere, and the resource doesn’t give me the training samples.
Stage 2: well, now I want to go to my final target, add to one more class, maybe alien bugs. In other words, this final NN Z can classify 5 classes after all ( cat, dog, human male, female, and alien bugs). Here, I do assume i have few training sample for the alien bugs. As you can tell, If I do can do stage 1, then this stage 2 become the standard transfer learning. I just train the last few layers of “combined A+B”.
Concatenate Hmm. looks so easy. lol. Sounds like I just concatenate the layers nn_a and nn_b together, that’s it ? so this nn_a + nn_b will truly do the work classify cat, dog, male, female very well? thank you
Essentially that shows three models being merged using Concatenate. The code is available at the linked page. In Keras, the Model class inherits from/extends the Layer class. So you can treat Model instances like Layer instances to do things like this. In Object-Oriented lingo, Model is-a Layer. Note that the inverse relationship is not true, that is, you cannot say Layer is-a Model. In the example at this page, the author just uses Layers directly, but it’s a trivial extension to make each one a Model, using either the Sequential or Functional API, and then use the Functional model to Concatenate. Usual restrictions on freezing parameters if you’re subsequently doing further training on a merged architecture.
Transfer learning generally involves freezing some or all layers of the existing model so that you don’t lose its training when it is incorporated into a new model. In the example linked above, all three channels are training on their respective inputs before being concatenated; it isn’t a transfer learning example. To adapt it to your hypothetical, each channel would be trained as a separate model, loaded as 3 separate models, trainable parameters frozen, then concatenated. Read more about Keras models and transfer learning here:
Key idea is this…Layers & models also feature a boolean attribute trainable . Its value can be changed.
