Hi,
Sorry for the silly question, but I am really unsure as to what I should be doing in line 35 - under the comment:
# set training to False to avoid keeping track of statistics in the batch norm layer
Any hints are greatly appreciated.
1 Like
The function base_model takes an optional named argument training, which can be specified as either True or False. Itâs a bit hard to deduce this from the documentation, because the MobilNetV2 class inherits it from one of its parent classes. But thatâs probably why they give you the comment: to save you the effort of digging through the layers of documentation.
It is quite confusing that the course expects you to understand and use the notation âtraining = trainingâ.
Itâs confusing because âtrainingâ is both the name of the argument label, and itâs a boolean variable that controls whether training is enabled.
Thank you both for your replies. From the comments, I understood that training should be set to False. My confusion is more with the first parameter when base_model is called:
# set training to False to avoid keeping track of statistics in the batch norm layer
x = base_model(None, training=False)
I assume that the None should be replaced with something to select the batch norm layer? However, I am not sure how to do this.
Thanks again!
The first input to base_model is simply the output of the previous operations (data augmentation followed by preprocessing). How else would it know what the input data is? The batch norm is happening within the layers of the model, whereas âtrainingâ is a state that applies to the whole model (do you need gradients computed or not).
That makes sense! Thank you for that!
Now I am having a new error thrown by the comparator function when comparing the summary with the expected output. It states that the number of layers is incorrect:
---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
<ipython-input-19-0346cb4bf847> in <module>
10 ['Dense', (None, 1), 1281, 'linear']] #linear is the default activation
11
---> 12 comparator(summary(model2), alpaca_summary)
13
14 for layer in summary(model2):
~/work/W2A2/test_utils.py in comparator(learner, instructor)
14 def comparator(learner, instructor):
15 if len(learner) != len(instructor):
---> 16 raise AssertionError(f"The number of layers in the model is incorrect. Expected: {len(instructor)} Found: {len(learner)}")
17 for a, b in zip(learner, instructor):
18 if tuple(a) != tuple(b):
AssertionError: The number of layers in the model is incorrect. Expected: 8 Found: 7
I printed the summary of the model and found the Sequential layer is missing (in addition to the output shapes of the Dropout and Dense layers being incorrect):
[['InputLayer', [(None, 160, 160, 3)], 0],
['TensorFlowOpLayer', [(None, 160, 160, 3)], 0],
['TensorFlowOpLayer', [(None, 160, 160, 3)], 0],
['Functional', (None, 5, 5, 1280), 2257984],
['AveragePooling2D', (None, 2, 2, 1280), 0],
['Dropout', (None, 2, 2, 1280), 0, 0.2],
['Dense', (None, 2, 2, 1), 1281, 'linear']]
I am pretty sure I havenât missed any lines in the provided skeleton code. As always, your help is greatly appreciated.
Aarjav
They show you the output that they are expecting in that test cell, right? So what is different about yours?
The differences between that and yours seem pretty clear. It looks like you did not include the data augmentation layer. Also the comment specifically told you to use âglobal average poolingâ, but you used just plain âaverage poolingâ.
I see! I called the data augmentation put didnât provide the input - I was looking at how the data_augmenter function was called to observe the output of the data_augmenter after the function was called. Thanks to your reply, I checked again and finally understood!
Sorry again for the onslaught of dumb questions, but now I need help in Exercise 3 with the accuracy metric. When running the test case, an assertion error is thrown:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-28-2a3573945f89> in <module>
3 assert type(optimizer) == tf.keras.optimizers.Adam, "This is not an Adam optimizer"
4 assert optimizer.lr == base_learning_rate / 10, "Wrong learning rate"
----> 5 assert metrics[0] == 'accuracy', "Wrong metric"
6
7 print('\033[92mAll tests passed!')
TypeError: 'Accuracy' object is not subscriptable
I looked through the available metrics in Tensorflow to make sure I am using the correct function (had a previous issue with using binary_crossentropy instead of BinaryCrossentropy). I am using the accuracy metric as shown in: tf.keras.metrics.Accuracy  | TensorFlow Core v2.7.0
Would you please provide some advice or suggestions on how to fix this?
Thank you!
Aarjav
It looks like you probably set the metrics input parameter just to be the string âAccuracyâ. Thatâs not correct. The metrics parameter is a python list containing strings. Hereâs an example of the difference:
thisIsMyString = 'Fred Flintstone'
thisIsMyListOfStrings = ['Fred', 'Barney', 'Wilma', 'Dino']
I see. Just for my understanding, is metrics a list because of a Tensor Flow thing? I didnât find any documentation which suggests that it would be a list (aside from the assert statement itself suggesting that it should be a list).
Also, I am really sorry, but now the assert statement checking for the correct metric states that the metric I used is incorrect. So, I printed what the metric was:
<tensorflow.python.keras.metrics.Accuracy object at 0x7f4e4c17b450>
When doing the check as in the assert statement:
print(metrics[0]=='accuracy')
The output is False
So, I thought maybe I should name the metric instance âaccuracyâ to match the test, but that, of course, did not change anything because the metric is still <tensorflow.python.keras.metrics.Accuracy object at 0x7f4e4c17b450>
Searching for another alias of the accuracy metric didnât lead me anywhere either.
As always I really appreciate your help and patience!
Thank you
The short answer is, âYes, itâs a TensorFlow thing.â Or at least a Keras thing.
According to the documentation [https://keras.io/api/metrics/]
The compile() method takes a metrics argument, which is a list of metrics:
model.compile(
optimizer='adam',
loss='mean_squared_error',
metrics=[
metrics.MeanSquaredError(),
metrics.AUC(),
]
)
And âŚ
All built-in metrics may also be passed via their string identifier (in this case, default constructor argument values are used, including a default metric name):
model.compile(
optimizer='adam',
loss='mean_squared_error',
metrics=[
'MeanSquaredError',
'AUC',
]
)
FinallyâŚ
To track metrics under a specific name, you can pass the name argument to the metric constructor:
model.compile(
optimizer='adam',
loss='mean_squared_error',
metrics=[
metrics.MeanSquaredError(name='my_mse'),
metrics.AUC(name='my_auc'),
]
)
2 Likes
Thanks @ai_curious ! That helped clear things up a bit. Regarding tracking metrics under a specific name, I did assign a name to the accuracy metric as I mentioned in a previous reply however, that still doesnât pass test in assert line.
Sorry to be posting my code, I will edit and remove once this is solved. I did the following:
# Use accuracy as evaluation metric
metrics=//Removed to avoid sharing answers
And to check the metric and test the use of correct metric:
print(metrics[0])
print(metrics[0] == 'accuracy')
The output of these print statements is (as before):
<tensorflow.python.keras.metrics.Accuracy object at 0x7f1cbc7f53d0>
False
Just to make sure I am not going crazy, I restarted the kernel to run everything and it still results in the same problem so I definitely am doing something wrong.
Excited to find a hint or suggestion that will reveal the error in my ways,
Aarjav
Your second print statement is asserting equivalency between the Accuracy object and a string literal. False is the correct evaluation of that comparison.
Hi,
Interesting, this is the exact same condition being used to test whether or not the metric is correct by the grader. If my output is correct but the condition in the print statement is wrong, then there is no way for anyone to pass this assignmentâŚ
For reference, this is the condition being used to test if the metric being used is correct:
assert metrics[0] == 'accuracy', "Wrong metric"
And this is the output from the above test case using the metric as I posted earlier:
---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
<ipython-input-30-2a3573945f89> in <module>
3 assert type(optimizer) == tf.keras.optimizers.Adam, "This is not an Adam optimizer"
4 assert optimizer.lr == base_learning_rate / 10, "Wrong learning rate"
----> 5 assert metrics[0] == 'accuracy', "Wrong metric"
6
7 print('\033[92mAll tests passed!')
AssertionError: Wrong metric
@ai_curious thanks to your last message, I went back and looked over the whole notebook. Turns out calling the TensorFlow metrics is not the way that is expected by the grader. It wanted me to set metrics as a list of string as @paulinpaloalto hinted. I guess I was somehow reading too much in the documentation rather than just looking at the notebook.
I am curious how using the accuracy metric in this way is preferred, in this case, over calling the accuracy metric as shown in tf.keras.metrics.Accuracy  | TensorFlow Core v2.7.0 but itâs all sorted now.
Thanks for your help everyone. I got there in the end 
Aarjav
2 Likes
The second block pasted above from the TF metrics page says
Apparently the course developers are forcing you to use this approach in order to make writing the grader assertions simpler. Glad you got it.
Yep, I just didnât realise that that was the issue with my code. Anyways, all good now.
I wanted to link in a related thread where fellow learner @Leonard_Truscello made a valuable contribution by reporting that using the string name syntax option allows the model.compile() function to pick a metric based on the loss function, thus enforcing consistency.
[Transfer Learning with MobileNetV2 Error: 'Accuracy' object is not subscriptable - #8 by Leonard_Truscello]
For others reading this thread after hitting either the âAccuracy object is not subscriptableâ runtime error or failing the model[0] == âaccuracyâ test assertion, take a look there.
2 Likes
About the original error in the post, I made sure that I am using the correct pooling layer but the test still confused it as AveragePooling2D, even after restarting the kernal. I ignored it and at the end got 100% from the grader.