Class Activation Map

Course Q&A TensorFlow: Advanced Techniques Specialization Advanced Computer Vision with TensorFlow
1

I’m working on an image classification project using the VGG16 model along with 3 Dense layers when I followed the instructions for C3_W4_Lab_1_FashionMNIST-CAM.ipynb I have two problems first one was specifying model input and output I have to use the input and output for the vgg model and the top classifier Dense layers as follows

cam_model  = Model(inputs=[base_model.input,output_model.input],outputs=[base_model.layers[-2].output,output_model.layers[-1].output])
cam_model.summary()

base_model: is the VGG16 model
output_model: is the top classifier that I have created

Model: "model_5"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            [(None, 224, 224, 3) 0                                            
__________________________________________________________________________________________________
block1_conv1 (Conv2D)           (None, 224, 224, 64) 1792        input_1[0][0]                    
__________________________________________________________________________________________________
block1_conv2 (Conv2D)           (None, 224, 224, 64) 36928       block1_conv1[0][0]               
__________________________________________________________________________________________________
block1_pool (MaxPooling2D)      (None, 112, 112, 64) 0           block1_conv2[0][0]               
__________________________________________________________________________________________________
block2_conv1 (Conv2D)           (None, 112, 112, 128 73856       block1_pool[0][0]                
__________________________________________________________________________________________________
block2_conv2 (Conv2D)           (None, 112, 112, 128 147584      block2_conv1[0][0]               
__________________________________________________________________________________________________
block2_pool (MaxPooling2D)      (None, 56, 56, 128)  0           block2_conv2[0][0]               
__________________________________________________________________________________________________
block3_conv1 (Conv2D)           (None, 56, 56, 256)  295168      block2_pool[0][0]                
__________________________________________________________________________________________________
block3_conv2 (Conv2D)           (None, 56, 56, 256)  590080      block3_conv1[0][0]               
__________________________________________________________________________________________________
block3_conv3 (Conv2D)           (None, 56, 56, 256)  590080      block3_conv2[0][0]               
__________________________________________________________________________________________________
block3_pool (MaxPooling2D)      (None, 28, 28, 256)  0           block3_conv3[0][0]               
__________________________________________________________________________________________________
vgg16_input (InputLayer)        [(None, 224, 224, 3) 0                                            
__________________________________________________________________________________________________
block4_conv1 (Conv2D)           (None, 28, 28, 512)  1180160     block3_pool[0][0]                
__________________________________________________________________________________________________
vgg16 (Functional)              (None, 7, 7, 512)    14714688    vgg16_input[0][0]                
__________________________________________________________________________________________________
block4_conv2 (Conv2D)           (None, 28, 28, 512)  2359808     block4_conv1[0][0]               
__________________________________________________________________________________________________
flatten_1 (Flatten)             (None, 25088)        0           vgg16[0][0]                      
__________________________________________________________________________________________________
block4_conv3 (Conv2D)           (None, 28, 28, 512)  2359808     block4_conv2[0][0]               
__________________________________________________________________________________________________
dense_3 (Dense)                 (None, 128)          3211392     flatten_1[0][0]                  
__________________________________________________________________________________________________
block4_pool (MaxPooling2D)      (None, 14, 14, 512)  0           block4_conv3[0][0]               
__________________________________________________________________________________________________
dropout_2 (Dropout)             (None, 128)          0           dense_3[0][0]                    
__________________________________________________________________________________________________
block5_conv1 (Conv2D)           (None, 14, 14, 512)  2359808     block4_pool[0][0]                
__________________________________________________________________________________________________
dense_4 (Dense)                 (None, 64)           8256        dropout_2[0][0]                  
__________________________________________________________________________________________________
block5_conv2 (Conv2D)           (None, 14, 14, 512)  2359808     block5_conv1[0][0]               
__________________________________________________________________________________________________
dropout_3 (Dropout)             (None, 64)           0           dense_4[0][0]                    
__________________________________________________________________________________________________
block5_conv3 (Conv2D)           (None, 14, 14, 512)  2359808     block5_conv2[0][0]               
__________________________________________________________________________________________________
dense_5 (Dense)                 (None, 1)            65          dropout_3[0][0]                  
==================================================================================================
Total params: 32,649,089
Trainable params: 17,934,401
Non-trainable params: 14,714,688 ```

when I came to predict the results on my test data the code produces an error that says

ValueError: Layer model_5 expects 2 input(s), but it received 1 input tensors. Inputs received: [<tf.Tensor 'IteratorGetNext:0' shape=(None, 224, 224, 3) dtype=float32>]```

what should I can to solve this problem and visualize what my model have learnt ?

2

Hello, husseinmleng!

With the lines of code you posted I can’t know exactly why the error is occurring. But it seems the reason would be whatever line you used to try to get the prediction.

This line you posted here… :

cam_model  = Model(inputs=[base_model.input,output_model.input],outputs=[base_model.layers[-2].output,output_model.layers[-1].output])

… is defining the model to have two inputs, but I would guess whatever line you used to make the prediction is only sending one input through the model, which would generate this error.

Seems you are trying to pass an image, when the model you defined requires an image plus a tensor in the shape of the output_model.input

1 Like
3

Visulaization_&_Interpretablity.ipynb (41.8 KB)
Here is the whole code.
what I’m trying to do is to visulaize what vgg16 model if i specified the input for the model to be only vgg16 input as follows :

inputs = Input((224,224,3))
cam_model  = Model(inputs=[base_model.input],outputs=[base_model.layers[-2].output,output_model.layers[-1].output])
cam_model.summary()

the following error appears because they are two models

ValueError: Graph disconnected: cannot obtain value for tensor KerasTensor(type_spec=TensorSpec(shape=(None, 224, 224, 3), dtype=tf.float32, name='vgg16_input'), name='vgg16_input', description="created by layer 'vgg16_input'") at layer "vgg16". The following previous layers were accessed without issue: ['block1_conv1', 'block1_conv2', 'block1_pool', 'block2_conv1', 'block2_conv2', 'block2_pool', 'block3_conv1', 'block3_conv2', 'block3_conv3', 'block3_pool']

I want to input the image to the model and get the prediction of the class and visualize it the same as the lab.
is that possible in my case ?

4

If you define your model with two inputs, then when it’s time to predict, pass a list of two tensors, for example:

cam_model.predict([X_test, X_test_2])

The new definition you just posted, with just one input is gonna throw an error unless you also remove output_model.layers[-1].output from the outputs. That is because cam_model will then be trying to connect base_model.input to output_model.layers[-1].output, but not finding a connected path.

5

As for this question I’m afraid I don’t 100% understand your query. It is indeed possible to get visualizations from layers outputs (if I’m understanding your intentions correctly), but some extra work might be required. For instance, if a layer’s output is in the shape of [1, a, b, 3], then it might be simple enough to use packages such as pyplot to read that as an image. For layers that output in the shape of, for instance, [1, a, b, 256], which is in a much higher dimensionality, it’s not as straight forward.

6

the problem here is that I don’t know what the values of the second tensor should be.
it is an image classification task so the only input that I have Is the input images?