# Questions about transfer learning in "Transfer\_learning\_with\_MobileNet\_v1"

**URL:** <https://community.deeplearning.ai/t/questions-about-transfer-learning-in-transfer-learning-with-mobilenet-v1/235603>\
**Category:** Convolutional Neural Networks\
**Tags:** coursera-platform\
**Created:** [November 27, 2022, 8:38pm UTC](https://community.deeplearning.ai/t/questions-about-transfer-learning-in-transfer-learning-with-mobilenet-v1/235603 "2022-11-27T20:38:28Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![sunson29](https://avatars.discourse-cdn.com/v4/letter/s/e495f1/32.png) [@sunson29](https://community.deeplearning.ai/u/sunson29)\
**Post date:** [November 27, 2022, 8:38pm UTC](https://community.deeplearning.ai/t/questions-about-transfer-learning-in-transfer-learning-with-mobilenet-v1/235603/1 "2022-11-27T20:38:28Z")

</div>

Hi guys,

I already completed the hw of `Transfer_learning_with_MobileNet_v1`. But I still have some questions. Thanks for your time in advance.

My understanding: to use transfer learning, I download a premade NN work (assume 10 layers total), I just modify the last 2 layers (fully connected layers) for my new target. This also means, I only train the last 2 modified layers. The original first 8 layers are “frozen”. If my understanding is correct. My questions are below:

Q1. In the section of `Exercise 2 - alpaca_model`, since we have

```auto
def alpaca_model(image_shape=IMG_SIZE, data_augmentation=data_augmenter()):
    
    input_shape = image_shape + (3,)
    
    ### START CODE HERE
    
    base_model = tf.keras.applications.MobileNetV2(input_shape=,
                                                   include_top=, # <== Important!!!!
                                                   weights='') # From imageNet
    
    # freeze the base model by making it non trainable
    base_model.trainable = False
    # create the input layer (Same as the imageNetv2 input size)
    inputs = Input...
    # apply data augmentation to the inputs
    x = augmentation....
    # data preprocessing using the same weights the model was trained on
    x = preprocess_input...
    # set training to False to avoid keeping track of statistics in the batch norm layer
    x = base_model...
    # add the new Binary classification layers
    # use global avg pooling to summarize the info in each channel
    x = GlobalAveragePooling2D...
    # include dropout with probability of 0.2 to avoid overfitting
    x = Dropout ...    
    # use a prediction layer with one neuron (as a binary classifier only needs one)
    outputs = Dense...
    
    ### END CODE HERE
    model = tf.keras.Model(inputs, outputs)
    return model

```

Then we use this model to train

```auto
initial_epochs = 5
history = model2.fit(train_dataset, validation_data=validation_dataset, epochs=initial_epochs)

```

My question is, you already have `base_model.trainable = False.` What are we training here? My understanding here is, this whole NN is NOT trainable, even the newly added layers. Is this right or wrong?

After this section, we start to do Fine-tuning. My understanding here is, I need to unfreeze only the newly added fully connected layers, and train them only (layers before those newly added layers are still frozen) .  
I want to double-check my understanding of the code below:

```auto
base_model = model2.layers[4]
base_model.trainable = True

```

Q2. what’s meaning of trainable = True for layers[4] ?

```auto
# Freeze all the layers before the `fine_tune_at` layer
for layer in base_model.layers[:fine_tune_at]:
    layer.trainable = True

```

Q3. Freeze or un freeze layers ? The running index is layer, and it runs from 1st to fine\_tune\_at, and the trainable = True. my understanding of this code is, un-freeze the layers from 1st to fine\_tune\_at layers. But I need to un freeze the layers after fine\_tune\_at, right?

I do aware I need to improve my python skills ☹ Thanks for your time again.

---

<div class="post-metadata">

**Author:** ![TMosh](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/tmosh/32/9321_2.png) [@TMosh](https://community.deeplearning.ai/u/TMosh)\
**Post date:** [November 27, 2022, 9:19pm UTC](https://community.deeplearning.ai/t/questions-about-transfer-learning-in-transfer-learning-with-mobilenet-v1/235603/3 "2022-11-27T21:19:07Z")

</div>

Note, I have deleted my previous reply.

---

<div class="post-metadata">

**Author:** ![Juan\_Olano](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/juan_olano/32/27125_2.png) [@Juan\_Olano](https://community.deeplearning.ai/u/Juan_Olano)\
**Post date:** [November 27, 2022, 9:27pm UTC](https://community.deeplearning.ai/t/questions-about-transfer-learning-in-transfer-learning-with-mobilenet-v1/235603/4 "2022-11-27T21:27:24Z")

</div>

Hi @sunson29 ,

I just replied to another learner about a similar question. I’d like to share that answer in hopes that it will shed light to your question:

> [@What is the difference between pre-training and fine tuning in transfer learning?](https://community.deeplearning.ai/t/what-is-the-difference-between-pre-training-and-fine-tuning-in-transfer-learning/235510/4):
>
> Hi @Shivam_sharma_MED_07 , Regarding pre-trained models: You can find and download from the internet models that are ‘ready to use’ - these are pre-trained models. Using these pre-trained models will save us a lot of time if the model was trained for my exact objective. For example, if I want to classify cats in general, I can download a model pre-trained in cats, or even a model pre-trained in domestic animals that include cats. Now, sometimes my need is more specific and I cannot find a pre…

Juan

---

<div class="post-metadata">

**Author:** ![sunson29](https://avatars.discourse-cdn.com/v4/letter/s/e495f1/32.png) [@sunson29](https://community.deeplearning.ai/u/sunson29)\
**Post date:** [November 27, 2022, 10:02pm UTC](https://community.deeplearning.ai/t/questions-about-transfer-learning-in-transfer-learning-with-mobilenet-v1/235603/5 "2022-11-27T22:02:22Z")

</div>

Hi Juan,

Thanks for the link. You explained it very well !!! I saved as my note 🙂 I think my understanding was correct.

However, if I may, could you explain more on the side of the code of my question? I wish to 100% synchronize my understanding with the code. for example, in Q1, if trainable = False, what are we training later from model.fit? In Q2, Which the layers are frozen after all?

Juan, could you check my questions again? thank you for your time!

---

<div class="post-metadata">

**Author:** ![rmwkwok](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/rmwkwok/32/126977_2.png) [@rmwkwok](https://community.deeplearning.ai/u/rmwkwok)\
**Post date:** [November 27, 2022, 10:10pm UTC](https://community.deeplearning.ai/t/questions-about-transfer-learning-in-transfer-learning-with-mobilenet-v1/235603/6 "2022-11-27T22:10:33Z")

</div>

Hello @sunson29,

Let’s examine your quoted code while discussing your question:

> [@sunson29](#):
>
> ```auto
> # use a prediction layer with one neuron (as a binary classifier only needs one)
> outputs = Dense...
> 
> ```

This is the answer for your first question. When you define `alpaca_model`, you added this very `Dense` layer which is by default **trainable**. Although the `base_model` is not trainable, the `Dense` layer **IS** trainable. Therefore:

> [@sunson29](#):
>
> What are we training here?

The `Dense` layer.

> [@sunson29](#):
>
> My understanding here is, this whole NN is NOT trainable, even the newly added layers. Is this right or wrong?

Wrong.

> [@sunson29](#):
>
> I need to unfreeze only the newly added fully connected layers, and train them only (layers before those newly added layers are still frozen) .

You don’t need to unfreeze the `Dense` layer because it is trainable by default.

> [@sunson29](#):
>
> I want to double-check my understanding of the code below:
> 
> ```auto
> base_model = model2.layers[4]
> base_model.trainable = True
> 
> ```

This is how you can set a layer to be trainable. However, you need to make sure `4` is the right layer index you want to modify. Also, after you get the correct layer index, before assigning `True` to its trainable attribute, I suggest you to print the value of it to see that it is by default set to `True` already.

> [@sunson29](#):
>
> ```auto
> # Freeze all the layers before the `fine_tune_at` layer
> for layer in base_model.layers[:fine_tune_at]:
> layer.trainable = True
> 
> ```

I think setting the right value for `layer.trainable` is your assignment ;), what is the correct value of it if your objective is to _ **Freeze all the layers before the `fine_tune_at` layer** _ ?

> [@sunson29](#):
>
> my understanding of this code is, un-freeze the layers from 1st to fine\_tune\_at layers. But I need to un freeze the layers after fine\_tune\_at, right?

This is not about the understanding of the code. This is about the understanding of the requirement of the exercise. I strongly suggest you to read again the exercise 3 description right right right before the code cell of Exercise 3. It will answer your question!

Cheers,  
Raymond

---

<div class="post-metadata">

**Author:** ![sunson29](https://avatars.discourse-cdn.com/v4/letter/s/e495f1/32.png) [@sunson29](https://community.deeplearning.ai/u/sunson29)\
**Post date:** [December 1, 2022, 9:39pm UTC](https://community.deeplearning.ai/t/questions-about-transfer-learning-in-transfer-learning-with-mobilenet-v1/235603/7 "2022-12-01T21:39:40Z")

</div>

Hi @rmwkwok

Sorry about this late reply. I am very busy with my work, especially at end of the year.

For the Q1, thanks for the reply in detail. Yes, I understand now.  
For my Q2, no. I am still stuck here. Allow me to demonstrate my question here:

step 1: unfreeze the base model

```auto
base_model = model2.layers[4]
base_model.trainable = True

```

we pick this layer 4, which is the original loaded network. Then unfreeze it. This means, all layers in base\_model ( or model2.layers[4]) are trainable. I checked the code. Yes, I agree and I do see all layers[4] trainable=True. after the code above.

Step 2: However, the next step: set a layer to fine-tune from, then re-freeze all the layers before it. I think the code is not doing what it described.

```auto
# Fine-tune from this layer onwards
fine_tune_at = 120
# Freeze all the layers before the `fine_tune_at` layer
for layer in base_model.layers[:fine_tune_at]:
    layer.trainable = True

```

I ran the code above, and checked those layers.trainable in base\_model. All I see is that they are ALL trainable. I think the True here should be the False. So that, the layers [0 to 119].trainable will be == False (re-freeze all the layers before it).  
In other words, if I do the code as below, I saw layers from [0 to 119] are freeze (trainable = False), 120 to end are trainable = True

```auto
for layer in base_model.layers[:fine_tune_at]:
    layer.trainable = True

```

@rmwkwok could you enlighten me again, like the way you do before, step by step. Thank you

---

<div class="post-metadata">

**Author:** ![rmwkwok](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/rmwkwok/32/126977_2.png) [@rmwkwok](https://community.deeplearning.ai/u/rmwkwok)\
**Post date:** [December 2, 2022, 2:12am UTC](https://community.deeplearning.ai/t/questions-about-transfer-learning-in-transfer-learning-with-mobilenet-v1/235603/8 "2022-12-02T02:12:24Z")

</div>

> [@sunson29](#):
>
> I think the True here should be the False.

I agree. Please note that initially the assignment code is like this

```auto
for layer in base_model.layers[:fine_tune_at]:
    layer.trainable = None # <-- it is a None initially

```

so it is your exercise work to replace `None` with a correct answer, and I agree with your argument on what `None` should be replaced with.

> [@sunson29](#):
>
> could you enlighten me again, like the way you do before, step by step. Thank you

I think you have pretty much explained everything yourself 😉

Raymond

---

<div class="post-metadata">

**Author:** ![sunson29](https://avatars.discourse-cdn.com/v4/letter/s/e495f1/32.png) [@sunson29](https://community.deeplearning.ai/u/sunson29)\
**Post date:** [December 2, 2022, 7:11pm UTC](https://community.deeplearning.ai/t/questions-about-transfer-learning-in-transfer-learning-with-mobilenet-v1/235603/9 "2022-12-02T19:11:17Z")

</div>

hey @rmwkwok omg… I think I tricked myself after all. I must accidentally change the `layer.trainable = None` to `layer.trainable = True`, then I believed this is given the initial code… Thank you for your help!

@rmwkwok one more question here please, but very simple.

```auto
base_model = tf.keras.applications.MobileNetV2(input_shape=IMG_SHAPE,
                                               include_top=True,
                                               weights='imagenet')

```

In the code above, if include\_top is False, the last 2 layers (in this MobileNetV2 case) will NOT be included and I understand that. My question is, which code I can use to check the layers whether belong to top or not ? I am expecting to see some code like:

```auto
base_model.layers[layerNumber].thisistop = True or False

```

Because if the premade NN is changed, I need to know how many last layers are considered as “top”. I do know I could just do a comparison between true and false, but I hope that if there is a simple code to do that. Thank you

---

<div class="post-metadata">

**Author:** ![rmwkwok](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/rmwkwok/32/126977_2.png) [@rmwkwok](https://community.deeplearning.ai/u/rmwkwok)\
**Post date:** [December 3, 2022, 4:19am UTC](https://community.deeplearning.ai/t/questions-about-transfer-learning-in-transfer-learning-with-mobilenet-v1/235603/10 "2022-12-03T04:19:54Z")

</div>

No worries @sunson29😉

Unfortunately I don’t think there is such an attribute to find out if it is part of the “top” or not. Indeed, comparing True and False as you said is one way, while what I have done before replying to you is that:

1. [go to the doc](https://www.tensorflow.org/api_docs/python/tf/keras/applications/mobilenet_v2/MobileNetV2)
2. “View source on Github”
3. Found [this part which actually reacts to “include\_top”](https://github.com/keras-team/keras/blob/e6784e4302c7b8cd116b74a784f4b78d60e83c26/keras/applications/mobilenet_v2.py#L431-L442)

There is no simpler way 😉

Cheers,  
Raymond

---

<div class="post-metadata">

**Author:** ![sunson29](https://avatars.discourse-cdn.com/v4/letter/s/e495f1/32.png) [@sunson29](https://community.deeplearning.ai/u/sunson29)\
**Post date:** [December 3, 2022, 9:33pm UTC](https://community.deeplearning.ai/t/questions-about-transfer-learning-in-transfer-learning-with-mobilenet-v1/235603/11 "2022-12-03T21:33:30Z")

</div>

got it. thanks again!

---

<div class="post-metadata">

**Author:** ![rmwkwok](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/rmwkwok/32/126977_2.png) [@rmwkwok](https://community.deeplearning.ai/u/rmwkwok)\
**Post date:** [December 3, 2022, 11:28pm UTC](https://community.deeplearning.ai/t/questions-about-transfer-learning-in-transfer-learning-with-mobilenet-v1/235603/12 "2022-12-03T23:28:15Z")

</div>

You are welcome @sunson29 !