Transfer Learning - include_top

Hi! I’m making an app that uses a CNN to classify math symbols. If I used some network, say EfficientNet, and set include_top = false, does that mean it retrains all the weights for the whole network, or just the last layer?

Thanks!

We’re doing object oriented programming here and we’re using all this complex code that someone else wrote. I would start by reading the documentation. Or here’s a StackExchange thread you get by googling “keras model include_top”.

What you learn by reading that is that “include_top” has to do with whether you include the last few fully connected layers of the pretrained network that you are starting from. Whether you want that or not depends on what your specific goals are. If you took DLS C4, it would be worth going back and reviewing all this in the Transfer Learning with MobilNet assignment in Week 2. There they start with MobilNetv2, which is a model that has 1000 output classes. But they want to “repurpose” it to specifically identify alpaca versus not alpaca, meaning that the output layer needs to be sigmoid for a binary classification, instead of a softmax layer with 1000 classes, so they use include_top = False.

Then the question of how much of the pretrained network you “freeze” and use as is and how much of it you retrain is a separate topic. That is also covered in the MobilNet assignment: they show you how to tune where the “freeze” point is for the additional training.

1 Like

Well, that was my question.

Yes and I’ve pointed you to the place in the course where that subject is covered. I checked your forum activity history and it looks like you took DLS C4. Please have a look at the MobilNet assignment again to see how the layer retraining can be managed.

I no longer have access to the course.

Ah, ok, well, the best idea would have been to download the notebooks while you were taking the course, so that you could refer to them later. There are several relevant topics on the DLS FAQ Thread. I guess it’s a little late for now for this particular issue, but please keep that in mind in any future courses that you take.

So we’ll have to resort to the “google strategy”, as I demonstrated in my earlier reply with the StackExchange thread about “include_top”. This type of information is available on StackExchange and in the Keras docs and tutorials. I will try to do some searching and suggest some sites, but if you’re going to be operating in this space, you should be developing those skills yourself. The information is all out there, so you just have to spend a bit of effort to find it.

The Keras website has a lot of good information, including documentation of all the pretrained models they have, example code to do fine-tuning and tutorials about Transfer Learning. Here’s the top level Keras page for that, which starts with the catalog of all the models they have.

Here’s the next level down that is specific to the MobilNet models. That page in turn points to some other great resources:

Examples of image processing Transfer Learning use cases.

Transfer Learning and Fine-Tuning. That page looks like the one that will really address your questions about freezing and unfreezing layers. Please have a look! It’s basically an online version of a chapter from François Chollet’s (the author of Keras) book about Keras. I haven’t read it in detail yet, but just scanned through and it looks like it covers things thoroughly with lots of example code.

Thanks for the info.