Course 4 week 2 project 2

Course Q&A Deep Learning Specialization Convolutional Neural Networks
1

I have everything good except for a couple of the None values from the second model. Here is what i have

{moderator edit - solution code removed}

Can anyone give me a nudge in the right direction on these couple None values i still have?

5 Likes
2

I think you need to pass the input to the sequential model. The other mistake I find is that you have to pass the processed data to the base model.
I hope this helps you
Regards

1 Like
3

Im still having an issue with this line “x.add(preprocess_input(np.expand_dims(IMG_SHAPE, axis=0)))”. Should i not add it to the Sequential? The error i am getting is this traceback.
TypeError Traceback (most recent call last)
in
----> 1 model2 = alpaca_model(IMG_SIZE, data_augmentation)

in alpaca_model(image_shape, data_augmentation)
35 # data preprocessing using the same weights the model was trained on
36 # x = preprocess_input(np.expand_dims(IMG_SHAPE, axis=1))
—> 37 x.add(preprocess_input(np.expand_dims(IMG_SHAPE, axis=0)))
38
39 # set training to False to avoid keeping track of statistics in the batch norm layer

/opt/conda/lib/python3.7/site-packages/tensorflow/python/training/tracking/base.py in _method_wrapper(self, *args, **kwargs)
455 self._self_setattr_tracking = False # pylint: disable=protected-access
456 try:
→ 457 result = method(self, *args, **kwargs)
458 finally:
459 self._self_setattr_tracking = previous_value # pylint: disable=protected-access

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/sequential.py in add(self, layer)
180 raise TypeError('The added layer must be ’
181 'an instance of class Layer. ’
→ 182 'Found: ’ + str(layer))
183
184 tf_utils.assert_no_legacy_layers([layer])

TypeError: The added layer must be an instance of class Layer. Found: [[ 0.254902 0.254902 -0.9764706]]

1 Like
4

This is my current state of the code. I am so lost, i have been just trying different things to see if it works and i got myself super lost on what i am doing, please could i have some guidance.

{moderator edit - solution code removed}

1 Like
5

Ok so i reset the notebook and started over. I came up with this for the second section.
{moderator edit - solution code removed}

It is now not returning any error for the block that makes model2, but the cell after that with the tests throws on error on the comparator function

AttributeError Traceback (most recent call last)
in
10 [‘Dense’, (None, 1), 1281, ‘linear’]] #linear is the default activation
11
—> 12 comparator(summary(model2), alpaca_summary)
13
14 for layer in summary(model2):

~/work/W2A2/test_utils.py in summary(model)
29 result =
30 for layer in model.layers:
—> 31 descriptors = [layer.class.name, layer.output_shape, layer.count_params()]
32 if (type(layer) == Conv2D):
33 descriptors.append(layer.padding)

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer_v1.py in output_shape(self)
1652 “”"
1653 if not self._inbound_nodes:
→ 1654 raise AttributeError('The layer has never been called ’
1655 ‘and thus has no defined output shape.’)
1656 all_output_shapes = set(

AttributeError: The layer has never been called and thus has no defined output shape.
333

Please assist?

1 Like
6

ok i have derived that it is the preprocessing step and the step the follows where you use the base_model() variable in the MobileNetV2 model. Can anyone help me figure out how to write this line. This is my code as it is right now.

{moderator edit - solution code removed}

1 Like
7

Ok i have got it so the model trains. On the test i am getting test errors now. here is the code i have
.

{moderator edit - solution code removed}

any help. I post on here because once i do, I get somewhere better with the code. So its somewhat just so i can get code improvements. but also, i really am stuck on this and losing confidence

1 Like
8

Here is my current summary output

Layer (type) Output Shape Param #

input_9 (InputLayer) multiple 0

tf_op_layer_RealDiv_3 (Tenso [(None, 160, 160, 3)] 0

tf_op_layer_Sub_3 (TensorFlo [(None, 160, 160, 3)] 0

Avg_Pool (GlobalAveragePooli (None, 3) 0

dropout_5 (Dropout) (None, 3) 0

fc (Dense) (None, 1) 4

Total params: 4
Trainable params: 4
Non-trainable params: 0

1 Like
9

Hello @dbiber

I was stuck at this exercise and I found your question while searching online. These websites helped me figure out each line mistake,
Transfer Learning Guide: A Practical Tutorial With Examples for Images and Text in Keras - neptune.ai

Transfer learning and fine-tuning | TensorFlow Core

tf.keras.applications.resnet.preprocess_input | TensorFlow Core v2.5.0

Transfer learning and fine-tuning | TensorFlow Core

After solving it now and it works fine for me. I still can’t figure out many things so I can’t explain why yours is correct or wrong. But Shortly

  1. input_shape should be the corrected input_shape = image_shape + (3,) It is the same shape that you need to use in creating inputs

  2. x should be equated to data_augmentation as augmenter() is already called by calling alpaca_model(...) but still you need to apply it to the inputs and so inputs should be received as argument. >> This is strange for me as my understanding to how augmenter() was created it is not receiving any arguments so I don’t understand how this is done in coding in general

  3. for the batch norm layer to avoid its updates is really confusing to write it that way because as per my understanding the batch norm are many layers and the nearest explanation of that would be that layer that is number 4 from the output side. I have no clue why it is called batch norm layer. But simply add x

  4. don’t add specifications in any of the method that are not mentioned by the exercise like name="Avg_Pool" or activation = 'sigmoid' or name='fc'

  5. one of my mistakes that I applied (x) to the last Dense layer which is already applied one step later equating it to outputs.

I hope that helped and please if you have answers to the questions I asked, that would be helpful.

11 Likes
10

YES!!! It helped me solve it!!! thank you soo much @ mourka

1 Like
11

Hey, can you please tell us what are your questions exactly, so that we can help you out? Are these 5 points that you mentioned, your questions?

1 Like
12

I was just stuck on the problem and had searched and searched through forums to find the answers and it ended up leading me in directions that resulted in jme having to complete reset the assignment. But the 5 points mentioned above helped me to solve my issues. The links were exactly what i was searching for but had not found until posted above. I have this issue resolved

1 Like
13

Dear @Elemento, thank you so much for your initiative to help me understand :slight_smile:

My questions are only two that are inside the 5 points but here I am clearing them out:

  1. Why x is representing the batch norm layer?

set training to False to avoid keeping track of statistics in the batch norm layer

As far as I understand that the last batch layer is the 4th from the end of the network ,

Conv_1_bn (BatchNormalization) (None, 5, 5, 1280) 5120 Conv_1[0][0]

out_relu (ReLU) (None, 5, 5, 1280) 0 Conv_1_bn[0][0]

global_average_pooling2d_7 (Glo (None, 1280) 0 out_relu[0][0]

predictions (Dense) (None, 1000) 1281000 global_average_pooling2d_7[0][0]

which could be accessed as follows
x = base_model(base_model.layers[len(base_model.layers - 4)],training = False)

but seems not to be the case. so my question what is the batch norm layer here, and why it is represented by x?

  1. This is more of a python question, I know OOP coding as well as Matlab but I am a bit new with python. How come we are creating a method with no argument and then use an argument for it?

{moderator edit - solution code removed}

Then in the model we used
data_augmentation=data_augmenter()
x = data_augmentation(inputs)

Like what is inputs in the method of data_augmenter(). data_augmentation is an object that is created by tf.keras.Sequential(), where is inputs here?

2 Likes
14

About this point

  1. x should be equated to data_augmentation as augmenter() is already called by calling alpaca_model(...) but still you need to apply it to the inputs and so inputs should be received as argument. >> This is strange for me as my understanding to how augmenter() was created it is not receiving any arguments so I don’t understand how this is done in coding in general

The reason why you’ve got to pass some arguments is that you are passing arguments to what the method returns, not to the method itself. What the method does is to create a Sequential Model, add some layers and return that model, so when you are passing the arguments is to the sequential model and not to the method.

Take a look on how alpaca_model is defined.
When the code says “def alpaca_model(image_shape=IMG_SIZE, data_augmentation=data_augmenter()):”, it means that when calling alpaca_model, it will accept as arguments two variables image_shape, and data_augmentation. In the case you call alpaca_model without any arguments it will take the variable IMG_SIZE as the value for image_shape and what data_augmenter() returns as data_augmentation. This gives you the posibility to re use alpaca model using the method you already defined or to use it with any other custom data augmentation method you may define before calling alpaca_model (obviously you’ve got to pass that method to alpaca_model by doing something like “alpaca_model(data_augmentation=mycustommethod())”.
Hope this helps.
Regards

2 Likes
15

I guess there is a small confusion as far as the 1st question is concerned. Here, I am trying to state my points of view. If you find any of them to be wrong, you can let me know, I will re-think them.

  • The first thing to note is that the comment snippet you have mentioned set training to False to avoid keeping track of statistics in the batch norm layer is a part of the alpaca_model function, and the summary you have mentioned is of the base_model.
  • Now, in the alpaca_model function, we aren’t adding any additional batch normalization layers. We are just using x = base_model(x, training=False) to make sure that the batch normalization layers which are a part of the base_model don’t track the statistics when we are training our alpaca model.
  • Now, I am not sure what you meant by the question “Why x is representing the batch norm layer?”, cause in the alpaca_model, x is not representing any batch_norm layer as far as I can see. If you still have any confusion, do let me know :smiley:
2 Likes
16

ok @Elemento, you made me think more about what each variable represents,

I was confused that both steps are made to or not to update the weights of the layers in training,

  1. base_model.trainable = False >> not updating layers of the Deep Network
  2. x = base_model(x, training=False) >> not updating the network to be added to the deep network which is x

I think the second step here I am understanding it wrong.

I think my confusion lies in the meaning of The big model don’t track the statistics
As far as I understand the batch normalization step is done not between the layers but separately in each layer. Like we do convolution with several filters and then normalize over all the result of the convolutions done in the current layer.

Saying track indicates that it is between layers and using training=False is indicating not updating weights.

This is where I am confused, I wasn’t expecting to add x in the argument here,
x = base_model(x, training=False)

1 Like
17

@andrescristi thank you so much, I see my mistake now we are giving an argument to the output object, not to the method.
data_augmentation = data_augmenter()
x = data_augmentation(inputs)

1 Like
18

Let me add a few more points. They might help you understand

  • tf.keras.layers.BatchNormalization  |  TensorFlow Core v2.5.0 I would like you to keep this link open while reading these points
  • I am sure that you must be well aware that a BatchNorm layer doesn’t have weights as other layers. They have parameters like momentum and epsilon which are used for normalization across the batch of inputs.
  • base_model.trainable = False This code snippet tells the model to not update the weights of layers of the base_model like convolutional layers, dense layers, etc.
  • x = base_model(x, training=False) On the other hand this code snippet major motive is to include the base_model inside the alpaca_model, as that’s what we are trying to, removing the top layers of the base_model and adding a few custom layers.
  • training=False Now this argument just tells the model, that the BatchNorm layers inside the base_model should not track the statistics.
  • You don’t need to confuse with the training argument of the BatchNorm layer. These 2 are completely different.

I hope these points help you a little

4 Likes
19

Need Help!!!

{moderator edit - solution code removed}

error: unsupported operand type(s) for /=: ‘Sequential’ and 'float’

1 Like
20

@Elemento Thank you so much for your explanations. It did help for sure, you are helping me to go through some forgotten topics and looking deeply linking the tensor flow code to what was explained.

I found this in the link that I sent before,

base_model.trainable = False

freeze the base model layers so that they’re not updated during the training process.

x = base_model(x, training=False)

ensure that the base model is running in inference mode so that batch normalization layers are not updated during the fine-tuning stage

From the link you sent me,
During training (i.e. when using fit())
During inference (i.e. when using evaluate() or predict()

so If I am not mistaken and you can correct me if I removed training=False it would still give the same results in the quiz because we didn’t use either evaluate() or predict() (we didn’t use inference in the quiz only fit() and plotting the results of that).

1 Like