Error in ResNet50 (Assignment 1)

Course Q&A Deep Learning Specialization Convolutional Neural Networks
1

Link to Assignment #1: Coursera | Online Courses & Credentials From Top Educators. Join for Free | Coursera

Hi, I’m having issues with creating ResNet50 in Exercise #3. Both the identity_block and convolution_block functions test successfully. ResNet50 has been implemented based on the instructions and I have triple checked the values being passed to each of the functions called in the exercises.

The model is built in the next step with the summary being:
Total params: 23,354,246
Trainable params: 23,301,126
Non-trainable params: 53,120

But in the following step it fails to compare to the “instructor” version of the model with the following error where the bolded number shows the discrepancy:

Test failed
Expected value
[‘Conv2D’, (None, 8, 8, 128), 147584, ‘same’, ‘linear’, ‘RandomUniform’]
does not match the input value:
[‘Conv2D’, (None, 8, 8, 128), 65664, ‘same’, ‘linear’, ‘RandomUniform’]

What do the bold numbers mean and how can I use this information to find the source of the problem?

2

Those numbers are the number of “parameters” in the defined layer. In a Conv2D layer, that would be the weights and bias values of the filters. Your output shape ends up being the same, but perhaps that means the size of your filters is incorrect in that layer. It looks like they must be smaller than they should be and then the “same” padding compensates to give you the correct output shape.

3

Here are the total parameter numbers that I get with a model that passes the tests:

Total params: 23,600,006
Trainable params: 23,546,886
Non-trainable params: 53,120

They are a bit higher than yours, which is consistent with the results for that one layer.

4

@paulinpaloalto - Would you mind sharing your model summary so that I can compare it with mine to determine where the error is. As I mentioned, I’ve triple checked the inputs along the way and all of my numbers line up with the instructions. But I must be missing something somewhere.

5

Not sure whether it will be useful in this form, but here’s the result of running
print(model.summary()):

Model: "model"
__________________________________________________________________________________________________
 Layer (type)                   Output Shape         Param #     Connected to                     
==================================================================================================
 input_1 (InputLayer)           [(None, 64, 64, 3)]  0           []                               
                                                                                                  
 zero_padding2d (ZeroPadding2D)  (None, 70, 70, 3)   0           ['input_1[0][0]']                
                                                                                                  
 conv2d_24 (Conv2D)             (None, 32, 32, 64)   9472        ['zero_padding2d[0][0]']         
                                                                                                  
 batch_normalization_24 (BatchN  (None, 32, 32, 64)  256         ['conv2d_24[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_21 (Activation)     (None, 32, 32, 64)   0           ['batch_normalization_24[0][0]'] 
                                                                                                  
 max_pooling2d (MaxPooling2D)   (None, 15, 15, 64)   0           ['activation_21[0][0]']          
                                                                                                  
 conv2d_25 (Conv2D)             (None, 15, 15, 64)   4160        ['max_pooling2d[0][0]']          
                                                                                                  
 batch_normalization_25 (BatchN  (None, 15, 15, 64)  256         ['conv2d_25[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_22 (Activation)     (None, 15, 15, 64)   0           ['batch_normalization_25[0][0]'] 
                                                                                                  
 conv2d_26 (Conv2D)             (None, 15, 15, 64)   36928       ['activation_22[0][0]']          
                                                                                                  
 batch_normalization_26 (BatchN  (None, 15, 15, 64)  256         ['conv2d_26[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_23 (Activation)     (None, 15, 15, 64)   0           ['batch_normalization_26[0][0]'] 
                                                                                                  
 conv2d_27 (Conv2D)             (None, 15, 15, 256)  16640       ['activation_23[0][0]']          
                                                                                                  
 conv2d_28 (Conv2D)             (None, 15, 15, 256)  16640       ['max_pooling2d[0][0]']          
                                                                                                  
 batch_normalization_27 (BatchN  (None, 15, 15, 256)  1024       ['conv2d_27[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 batch_normalization_28 (BatchN  (None, 15, 15, 256)  1024       ['conv2d_28[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_7 (Add)                    (None, 15, 15, 256)  0           ['batch_normalization_27[0][0]', 
                                                                  'batch_normalization_28[0][0]'] 
                                                                                                  
 activation_24 (Activation)     (None, 15, 15, 256)  0           ['add_7[0][0]']                  
                                                                                                  
 conv2d_29 (Conv2D)             (None, 15, 15, 64)   16448       ['activation_24[0][0]']          
                                                                                                  
 batch_normalization_29 (BatchN  (None, 15, 15, 64)  256         ['conv2d_29[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_25 (Activation)     (None, 15, 15, 64)   0           ['batch_normalization_29[0][0]'] 
                                                                                                  
 conv2d_30 (Conv2D)             (None, 15, 15, 64)   36928       ['activation_25[0][0]']          
                                                                                                  
 batch_normalization_30 (BatchN  (None, 15, 15, 64)  256         ['conv2d_30[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_26 (Activation)     (None, 15, 15, 64)   0           ['batch_normalization_30[0][0]'] 
                                                                                                  
 conv2d_31 (Conv2D)             (None, 15, 15, 256)  16640       ['activation_26[0][0]']          
                                                                                                  
 batch_normalization_31 (BatchN  (None, 15, 15, 256)  1024       ['conv2d_31[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_8 (Add)                    (None, 15, 15, 256)  0           ['batch_normalization_31[0][0]', 
                                                                  'activation_24[0][0]']          
                                                                                                  
 activation_27 (Activation)     (None, 15, 15, 256)  0           ['add_8[0][0]']                  
                                                                                                  
 conv2d_32 (Conv2D)             (None, 15, 15, 64)   16448       ['activation_27[0][0]']          
                                                                                                  
 batch_normalization_32 (BatchN  (None, 15, 15, 64)  256         ['conv2d_32[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_28 (Activation)     (None, 15, 15, 64)   0           ['batch_normalization_32[0][0]'] 
                                                                                                  
 conv2d_33 (Conv2D)             (None, 15, 15, 64)   36928       ['activation_28[0][0]']          
                                                                                                  
 batch_normalization_33 (BatchN  (None, 15, 15, 64)  256         ['conv2d_33[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_29 (Activation)     (None, 15, 15, 64)   0           ['batch_normalization_33[0][0]'] 
                                                                                                  
 conv2d_34 (Conv2D)             (None, 15, 15, 256)  16640       ['activation_29[0][0]']          
                                                                                                  
 batch_normalization_34 (BatchN  (None, 15, 15, 256)  1024       ['conv2d_34[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_9 (Add)                    (None, 15, 15, 256)  0           ['batch_normalization_34[0][0]', 
                                                                  'activation_27[0][0]']          
                                                                                                  
 activation_30 (Activation)     (None, 15, 15, 256)  0           ['add_9[0][0]']                  
                                                                                                  
 conv2d_35 (Conv2D)             (None, 8, 8, 128)    32896       ['activation_30[0][0]']          
                                                                                                  
 batch_normalization_35 (BatchN  (None, 8, 8, 128)   512         ['conv2d_35[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_31 (Activation)     (None, 8, 8, 128)    0           ['batch_normalization_35[0][0]'] 
                                                                                                  
 conv2d_36 (Conv2D)             (None, 8, 8, 128)    147584      ['activation_31[0][0]']          
                                                                                                  
 batch_normalization_36 (BatchN  (None, 8, 8, 128)   512         ['conv2d_36[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_32 (Activation)     (None, 8, 8, 128)    0           ['batch_normalization_36[0][0]'] 
                                                                                                  
 conv2d_37 (Conv2D)             (None, 8, 8, 512)    66048       ['activation_32[0][0]']          
                                                                                                  
 conv2d_38 (Conv2D)             (None, 8, 8, 512)    131584      ['activation_30[0][0]']          
                                                                                                  
 batch_normalization_37 (BatchN  (None, 8, 8, 512)   2048        ['conv2d_37[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 batch_normalization_38 (BatchN  (None, 8, 8, 512)   2048        ['conv2d_38[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_10 (Add)                   (None, 8, 8, 512)    0           ['batch_normalization_37[0][0]', 
                                                                  'batch_normalization_38[0][0]'] 
                                                                                                  
 activation_33 (Activation)     (None, 8, 8, 512)    0           ['add_10[0][0]']                 
                                                                                                  
 conv2d_39 (Conv2D)             (None, 8, 8, 128)    65664       ['activation_33[0][0]']          
                                                                                                  
 batch_normalization_39 (BatchN  (None, 8, 8, 128)   512         ['conv2d_39[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_34 (Activation)     (None, 8, 8, 128)    0           ['batch_normalization_39[0][0]'] 
                                                                                                  
 conv2d_40 (Conv2D)             (None, 8, 8, 128)    147584      ['activation_34[0][0]']          
                                                                                                  
 batch_normalization_40 (BatchN  (None, 8, 8, 128)   512         ['conv2d_40[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_35 (Activation)     (None, 8, 8, 128)    0           ['batch_normalization_40[0][0]'] 
                                                                                                  
 conv2d_41 (Conv2D)             (None, 8, 8, 512)    66048       ['activation_35[0][0]']          
                                                                                                  
 batch_normalization_41 (BatchN  (None, 8, 8, 512)   2048        ['conv2d_41[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_11 (Add)                   (None, 8, 8, 512)    0           ['batch_normalization_41[0][0]', 
                                                                  'activation_33[0][0]']          
                                                                                                  
 activation_36 (Activation)     (None, 8, 8, 512)    0           ['add_11[0][0]']                 
                                                                                                  
 conv2d_42 (Conv2D)             (None, 8, 8, 128)    65664       ['activation_36[0][0]']          
                                                                                                  
 batch_normalization_42 (BatchN  (None, 8, 8, 128)   512         ['conv2d_42[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_37 (Activation)     (None, 8, 8, 128)    0           ['batch_normalization_42[0][0]'] 
                                                                                                  
 conv2d_43 (Conv2D)             (None, 8, 8, 128)    147584      ['activation_37[0][0]']          
                                                                                                  
 batch_normalization_43 (BatchN  (None, 8, 8, 128)   512         ['conv2d_43[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_38 (Activation)     (None, 8, 8, 128)    0           ['batch_normalization_43[0][0]'] 
                                                                                                  
 conv2d_44 (Conv2D)             (None, 8, 8, 512)    66048       ['activation_38[0][0]']          
                                                                                                  
 batch_normalization_44 (BatchN  (None, 8, 8, 512)   2048        ['conv2d_44[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_12 (Add)                   (None, 8, 8, 512)    0           ['batch_normalization_44[0][0]', 
                                                                  'activation_36[0][0]']          
                                                                                                  
 activation_39 (Activation)     (None, 8, 8, 512)    0           ['add_12[0][0]']                 
                                                                                                  
 conv2d_45 (Conv2D)             (None, 8, 8, 128)    65664       ['activation_39[0][0]']          
                                                                                                  
 batch_normalization_45 (BatchN  (None, 8, 8, 128)   512         ['conv2d_45[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_40 (Activation)     (None, 8, 8, 128)    0           ['batch_normalization_45[0][0]'] 
                                                                                                  
 conv2d_46 (Conv2D)             (None, 8, 8, 128)    147584      ['activation_40[0][0]']          
                                                                                                  
 batch_normalization_46 (BatchN  (None, 8, 8, 128)   512         ['conv2d_46[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_41 (Activation)     (None, 8, 8, 128)    0           ['batch_normalization_46[0][0]'] 
                                                                                                  
 conv2d_47 (Conv2D)             (None, 8, 8, 512)    66048       ['activation_41[0][0]']          
                                                                                                  
 batch_normalization_47 (BatchN  (None, 8, 8, 512)   2048        ['conv2d_47[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_13 (Add)                   (None, 8, 8, 512)    0           ['batch_normalization_47[0][0]', 
                                                                  'activation_39[0][0]']          
                                                                                                  
 activation_42 (Activation)     (None, 8, 8, 512)    0           ['add_13[0][0]']                 
                                                                                                  
 conv2d_48 (Conv2D)             (None, 4, 4, 256)    131328      ['activation_42[0][0]']          
                                                                                                  
 batch_normalization_48 (BatchN  (None, 4, 4, 256)   1024        ['conv2d_48[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_43 (Activation)     (None, 4, 4, 256)    0           ['batch_normalization_48[0][0]'] 
                                                                                                  
 conv2d_49 (Conv2D)             (None, 4, 4, 256)    590080      ['activation_43[0][0]']          
                                                                                                  
 batch_normalization_49 (BatchN  (None, 4, 4, 256)   1024        ['conv2d_49[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_44 (Activation)     (None, 4, 4, 256)    0           ['batch_normalization_49[0][0]'] 
                                                                                                  
 conv2d_50 (Conv2D)             (None, 4, 4, 1024)   263168      ['activation_44[0][0]']          
                                                                                                  
 conv2d_51 (Conv2D)             (None, 4, 4, 1024)   525312      ['activation_42[0][0]']          
                                                                                                  
 batch_normalization_50 (BatchN  (None, 4, 4, 1024)  4096        ['conv2d_50[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 batch_normalization_51 (BatchN  (None, 4, 4, 1024)  4096        ['conv2d_51[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_14 (Add)                   (None, 4, 4, 1024)   0           ['batch_normalization_50[0][0]', 
                                                                  'batch_normalization_51[0][0]'] 
                                                                                                  
 activation_45 (Activation)     (None, 4, 4, 1024)   0           ['add_14[0][0]']                 
                                                                                                  
 conv2d_52 (Conv2D)             (None, 4, 4, 256)    262400      ['activation_45[0][0]']          
                                                                                                  
 batch_normalization_52 (BatchN  (None, 4, 4, 256)   1024        ['conv2d_52[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_46 (Activation)     (None, 4, 4, 256)    0           ['batch_normalization_52[0][0]'] 
                                                                                                  
 conv2d_53 (Conv2D)             (None, 4, 4, 256)    590080      ['activation_46[0][0]']          
                                                                                                  
 batch_normalization_53 (BatchN  (None, 4, 4, 256)   1024        ['conv2d_53[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_47 (Activation)     (None, 4, 4, 256)    0           ['batch_normalization_53[0][0]'] 
                                                                                                  
 conv2d_54 (Conv2D)             (None, 4, 4, 1024)   263168      ['activation_47[0][0]']          
                                                                                                  
 batch_normalization_54 (BatchN  (None, 4, 4, 1024)  4096        ['conv2d_54[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_15 (Add)                   (None, 4, 4, 1024)   0           ['batch_normalization_54[0][0]', 
                                                                  'activation_45[0][0]']          
                                                                                                  
 activation_48 (Activation)     (None, 4, 4, 1024)   0           ['add_15[0][0]']                 
                                                                                                  
 conv2d_55 (Conv2D)             (None, 4, 4, 256)    262400      ['activation_48[0][0]']          
                                                                                                  
 batch_normalization_55 (BatchN  (None, 4, 4, 256)   1024        ['conv2d_55[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_49 (Activation)     (None, 4, 4, 256)    0           ['batch_normalization_55[0][0]'] 
                                                                                                  
 conv2d_56 (Conv2D)             (None, 4, 4, 256)    590080      ['activation_49[0][0]']          
                                                                                                  
 batch_normalization_56 (BatchN  (None, 4, 4, 256)   1024        ['conv2d_56[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_50 (Activation)     (None, 4, 4, 256)    0           ['batch_normalization_56[0][0]'] 
                                                                                                  
 conv2d_57 (Conv2D)             (None, 4, 4, 1024)   263168      ['activation_50[0][0]']          
                                                                                                  
 batch_normalization_57 (BatchN  (None, 4, 4, 1024)  4096        ['conv2d_57[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_16 (Add)                   (None, 4, 4, 1024)   0           ['batch_normalization_57[0][0]', 
                                                                  'activation_48[0][0]']          
                                                                                                  
 activation_51 (Activation)     (None, 4, 4, 1024)   0           ['add_16[0][0]']                 
                                                                                                  
 conv2d_58 (Conv2D)             (None, 4, 4, 256)    262400      ['activation_51[0][0]']          
                                                                                                  
 batch_normalization_58 (BatchN  (None, 4, 4, 256)   1024        ['conv2d_58[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_52 (Activation)     (None, 4, 4, 256)    0           ['batch_normalization_58[0][0]'] 
                                                                                                  
 conv2d_59 (Conv2D)             (None, 4, 4, 256)    590080      ['activation_52[0][0]']          
                                                                                                  
 batch_normalization_59 (BatchN  (None, 4, 4, 256)   1024        ['conv2d_59[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_53 (Activation)     (None, 4, 4, 256)    0           ['batch_normalization_59[0][0]'] 
                                                                                                  
 conv2d_60 (Conv2D)             (None, 4, 4, 1024)   263168      ['activation_53[0][0]']          
                                                                                                  
 batch_normalization_60 (BatchN  (None, 4, 4, 1024)  4096        ['conv2d_60[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_17 (Add)                   (None, 4, 4, 1024)   0           ['batch_normalization_60[0][0]', 
                                                                  'activation_51[0][0]']          
                                                                                                  
 activation_54 (Activation)     (None, 4, 4, 1024)   0           ['add_17[0][0]']                 
                                                                                                  
 conv2d_61 (Conv2D)             (None, 4, 4, 256)    262400      ['activation_54[0][0]']          
                                                                                                  
 batch_normalization_61 (BatchN  (None, 4, 4, 256)   1024        ['conv2d_61[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_55 (Activation)     (None, 4, 4, 256)    0           ['batch_normalization_61[0][0]'] 
                                                                                                  
 conv2d_62 (Conv2D)             (None, 4, 4, 256)    590080      ['activation_55[0][0]']          
                                                                                                  
 batch_normalization_62 (BatchN  (None, 4, 4, 256)   1024        ['conv2d_62[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_56 (Activation)     (None, 4, 4, 256)    0           ['batch_normalization_62[0][0]'] 
                                                                                                  
 conv2d_63 (Conv2D)             (None, 4, 4, 1024)   263168      ['activation_56[0][0]']          
                                                                                                  
 batch_normalization_63 (BatchN  (None, 4, 4, 1024)  4096        ['conv2d_63[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_18 (Add)                   (None, 4, 4, 1024)   0           ['batch_normalization_63[0][0]', 
                                                                  'activation_54[0][0]']          
                                                                                                  
 activation_57 (Activation)     (None, 4, 4, 1024)   0           ['add_18[0][0]']                 
                                                                                                  
 conv2d_64 (Conv2D)             (None, 4, 4, 256)    262400      ['activation_57[0][0]']          
                                                                                                  
 batch_normalization_64 (BatchN  (None, 4, 4, 256)   1024        ['conv2d_64[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_58 (Activation)     (None, 4, 4, 256)    0           ['batch_normalization_64[0][0]'] 
                                                                                                  
 conv2d_65 (Conv2D)             (None, 4, 4, 256)    590080      ['activation_58[0][0]']          
                                                                                                  
 batch_normalization_65 (BatchN  (None, 4, 4, 256)   1024        ['conv2d_65[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_59 (Activation)     (None, 4, 4, 256)    0           ['batch_normalization_65[0][0]'] 
                                                                                                  
 conv2d_66 (Conv2D)             (None, 4, 4, 1024)   263168      ['activation_59[0][0]']          
                                                                                                  
 batch_normalization_66 (BatchN  (None, 4, 4, 1024)  4096        ['conv2d_66[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_19 (Add)                   (None, 4, 4, 1024)   0           ['batch_normalization_66[0][0]', 
                                                                  'activation_57[0][0]']          
                                                                                                  
 activation_60 (Activation)     (None, 4, 4, 1024)   0           ['add_19[0][0]']                 
                                                                                                  
 conv2d_67 (Conv2D)             (None, 2, 2, 512)    524800      ['activation_60[0][0]']          
                                                                                                  
 batch_normalization_67 (BatchN  (None, 2, 2, 512)   2048        ['conv2d_67[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_61 (Activation)     (None, 2, 2, 512)    0           ['batch_normalization_67[0][0]'] 
                                                                                                  
 conv2d_68 (Conv2D)             (None, 2, 2, 512)    2359808     ['activation_61[0][0]']          
                                                                                                  
 batch_normalization_68 (BatchN  (None, 2, 2, 512)   2048        ['conv2d_68[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_62 (Activation)     (None, 2, 2, 512)    0           ['batch_normalization_68[0][0]'] 
                                                                                                  
 conv2d_69 (Conv2D)             (None, 2, 2, 2048)   1050624     ['activation_62[0][0]']          
                                                                                                  
 conv2d_70 (Conv2D)             (None, 2, 2, 2048)   2099200     ['activation_60[0][0]']          
                                                                                                  
 batch_normalization_69 (BatchN  (None, 2, 2, 2048)  8192        ['conv2d_69[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 batch_normalization_70 (BatchN  (None, 2, 2, 2048)  8192        ['conv2d_70[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_20 (Add)                   (None, 2, 2, 2048)   0           ['batch_normalization_69[0][0]', 
                                                                  'batch_normalization_70[0][0]'] 
                                                                                                  
 activation_63 (Activation)     (None, 2, 2, 2048)   0           ['add_20[0][0]']                 
                                                                                                  
 conv2d_71 (Conv2D)             (None, 2, 2, 512)    1049088     ['activation_63[0][0]']          
                                                                                                  
 batch_normalization_71 (BatchN  (None, 2, 2, 512)   2048        ['conv2d_71[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_64 (Activation)     (None, 2, 2, 512)    0           ['batch_normalization_71[0][0]'] 
                                                                                                  
 conv2d_72 (Conv2D)             (None, 2, 2, 512)    2359808     ['activation_64[0][0]']          
                                                                                                  
 batch_normalization_72 (BatchN  (None, 2, 2, 512)   2048        ['conv2d_72[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_65 (Activation)     (None, 2, 2, 512)    0           ['batch_normalization_72[0][0]'] 
                                                                                                  
 conv2d_73 (Conv2D)             (None, 2, 2, 2048)   1050624     ['activation_65[0][0]']          
                                                                                                  
 batch_normalization_73 (BatchN  (None, 2, 2, 2048)  8192        ['conv2d_73[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_21 (Add)                   (None, 2, 2, 2048)   0           ['batch_normalization_73[0][0]', 
                                                                  'activation_63[0][0]']          
                                                                                                  
 activation_66 (Activation)     (None, 2, 2, 2048)   0           ['add_21[0][0]']                 
                                                                                                  
 conv2d_74 (Conv2D)             (None, 2, 2, 512)    1049088     ['activation_66[0][0]']          
                                                                                                  
 batch_normalization_74 (BatchN  (None, 2, 2, 512)   2048        ['conv2d_74[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_67 (Activation)     (None, 2, 2, 512)    0           ['batch_normalization_74[0][0]'] 
                                                                                                  
 conv2d_75 (Conv2D)             (None, 2, 2, 512)    2359808     ['activation_67[0][0]']          
                                                                                                  
 batch_normalization_75 (BatchN  (None, 2, 2, 512)   2048        ['conv2d_75[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 activation_68 (Activation)     (None, 2, 2, 512)    0           ['batch_normalization_75[0][0]'] 
                                                                                                  
 conv2d_76 (Conv2D)             (None, 2, 2, 2048)   1050624     ['activation_68[0][0]']          
                                                                                                  
 batch_normalization_76 (BatchN  (None, 2, 2, 2048)  8192        ['conv2d_76[0][0]']              
 ormalization)                                                                                    
                                                                                                  
 add_22 (Add)                   (None, 2, 2, 2048)   0           ['batch_normalization_76[0][0]', 
                                                                  'activation_66[0][0]']          
                                                                                                  
 activation_69 (Activation)     (None, 2, 2, 2048)   0           ['add_22[0][0]']                 
                                                                                                  
 average_pooling2d (AveragePool  (None, 1, 1, 2048)  0           ['activation_69[0][0]']          
 ing2D)                                                                                           
                                                                                                  
 flatten (Flatten)              (None, 2048)         0           ['average_pooling2d[0][0]']      
                                                                                                  
 dense (Dense)                  (None, 6)            12294       ['flatten[0][0]']                
                                                                                                  
==================================================================================================
Total params: 23,600,006
Trainable params: 23,546,886
Non-trainable params: 53,120
__________________________________________________________________________________________________
6

You can find another format in the file outputs.py, as you can see from the next cell that includes to call to comparator.

7

Actually this gives an easier to parse version:

for layer in summary(model):
    print(layer)

Here’s my result:

['InputLayer', [(None, 64, 64, 3)], 0]
['ZeroPadding2D', (None, 70, 70, 3), 0, ((3, 3), (3, 3))]
['Conv2D', (None, 32, 32, 64), 9472, 'valid', 'linear', 'GlorotUniform']
['BatchNormalization', (None, 32, 32, 64), 256]
['Activation', (None, 32, 32, 64), 0]
['MaxPooling2D', (None, 15, 15, 64), 0, (3, 3), (2, 2), 'valid']
['Conv2D', (None, 15, 15, 64), 4160, 'valid', 'linear', 'GlorotUniform']
['BatchNormalization', (None, 15, 15, 64), 256]
['Activation', (None, 15, 15, 64), 0]
['Conv2D', (None, 15, 15, 64), 36928, 'same', 'linear', 'GlorotUniform']
['BatchNormalization', (None, 15, 15, 64), 256]
['Activation', (None, 15, 15, 64), 0]
['Conv2D', (None, 15, 15, 256), 16640, 'valid', 'linear', 'GlorotUniform']
['Conv2D', (None, 15, 15, 256), 16640, 'valid', 'linear', 'GlorotUniform']
['BatchNormalization', (None, 15, 15, 256), 1024]
['BatchNormalization', (None, 15, 15, 256), 1024]
['Add', (None, 15, 15, 256), 0]
['Activation', (None, 15, 15, 256), 0]
['Conv2D', (None, 15, 15, 64), 16448, 'valid', 'linear', 'RandomUniform']
['BatchNormalization', (None, 15, 15, 64), 256]
['Activation', (None, 15, 15, 64), 0]
['Conv2D', (None, 15, 15, 64), 36928, 'same', 'linear', 'RandomUniform']
['BatchNormalization', (None, 15, 15, 64), 256]
['Activation', (None, 15, 15, 64), 0]
['Conv2D', (None, 15, 15, 256), 16640, 'valid', 'linear', 'RandomUniform']
['BatchNormalization', (None, 15, 15, 256), 1024]
['Add', (None, 15, 15, 256), 0]
['Activation', (None, 15, 15, 256), 0]
['Conv2D', (None, 15, 15, 64), 16448, 'valid', 'linear', 'RandomUniform']
['BatchNormalization', (None, 15, 15, 64), 256]
['Activation', (None, 15, 15, 64), 0]
['Conv2D', (None, 15, 15, 64), 36928, 'same', 'linear', 'RandomUniform']
['BatchNormalization', (None, 15, 15, 64), 256]
['Activation', (None, 15, 15, 64), 0]
['Conv2D', (None, 15, 15, 256), 16640, 'valid', 'linear', 'RandomUniform']
['BatchNormalization', (None, 15, 15, 256), 1024]
['Add', (None, 15, 15, 256), 0]
['Activation', (None, 15, 15, 256), 0]
['Conv2D', (None, 8, 8, 128), 32896, 'valid', 'linear', 'GlorotUniform']
['BatchNormalization', (None, 8, 8, 128), 512]
['Activation', (None, 8, 8, 128), 0]
['Conv2D', (None, 8, 8, 128), 147584, 'same', 'linear', 'GlorotUniform']
['BatchNormalization', (None, 8, 8, 128), 512]
['Activation', (None, 8, 8, 128), 0]
['Conv2D', (None, 8, 8, 512), 66048, 'valid', 'linear', 'GlorotUniform']
['Conv2D', (None, 8, 8, 512), 131584, 'valid', 'linear', 'GlorotUniform']
['BatchNormalization', (None, 8, 8, 512), 2048]
['BatchNormalization', (None, 8, 8, 512), 2048]
['Add', (None, 8, 8, 512), 0]
['Activation', (None, 8, 8, 512), 0]
['Conv2D', (None, 8, 8, 128), 65664, 'valid', 'linear', 'RandomUniform']
['BatchNormalization', (None, 8, 8, 128), 512]
['Activation', (None, 8, 8, 128), 0]
['Conv2D', (None, 8, 8, 128), 147584, 'same', 'linear', 'RandomUniform']
['BatchNormalization', (None, 8, 8, 128), 512]
['Activation', (None, 8, 8, 128), 0]
['Conv2D', (None, 8, 8, 512), 66048, 'valid', 'linear', 'RandomUniform']
['BatchNormalization', (None, 8, 8, 512), 2048]
['Add', (None, 8, 8, 512), 0]
['Activation', (None, 8, 8, 512), 0]
['Conv2D', (None, 8, 8, 128), 65664, 'valid', 'linear', 'RandomUniform']
['BatchNormalization', (None, 8, 8, 128), 512]
['Activation', (None, 8, 8, 128), 0]
['Conv2D', (None, 8, 8, 128), 147584, 'same', 'linear', 'RandomUniform']
['BatchNormalization', (None, 8, 8, 128), 512]
['Activation', (None, 8, 8, 128), 0]
['Conv2D', (None, 8, 8, 512), 66048, 'valid', 'linear', 'RandomUniform']
['BatchNormalization', (None, 8, 8, 512), 2048]
['Add', (None, 8, 8, 512), 0]
['Activation', (None, 8, 8, 512), 0]
['Conv2D', (None, 8, 8, 128), 65664, 'valid', 'linear', 'RandomUniform']
['BatchNormalization', (None, 8, 8, 128), 512]
['Activation', (None, 8, 8, 128), 0]
['Conv2D', (None, 8, 8, 128), 147584, 'same', 'linear', 'RandomUniform']
['BatchNormalization', (None, 8, 8, 128), 512]
['Activation', (None, 8, 8, 128), 0]
['Conv2D', (None, 8, 8, 512), 66048, 'valid', 'linear', 'RandomUniform']
['BatchNormalization', (None, 8, 8, 512), 2048]
['Add', (None, 8, 8, 512), 0]
['Activation', (None, 8, 8, 512), 0]
['Conv2D', (None, 4, 4, 256), 131328, 'valid', 'linear', 'GlorotUniform']
['BatchNormalization', (None, 4, 4, 256), 1024]
['Activation', (None, 4, 4, 256), 0]
['Conv2D', (None, 4, 4, 256), 590080, 'same', 'linear', 'GlorotUniform']
['BatchNormalization', (None, 4, 4, 256), 1024]
['Activation', (None, 4, 4, 256), 0]
['Conv2D', (None, 4, 4, 1024), 263168, 'valid', 'linear', 'GlorotUniform']
['Conv2D', (None, 4, 4, 1024), 525312, 'valid', 'linear', 'GlorotUniform']
['BatchNormalization', (None, 4, 4, 1024), 4096]
['BatchNormalization', (None, 4, 4, 1024), 4096]
['Add', (None, 4, 4, 1024), 0]
['Activation', (None, 4, 4, 1024), 0]
['Conv2D', (None, 4, 4, 256), 262400, 'valid', 'linear', 'RandomUniform']
['BatchNormalization', (None, 4, 4, 256), 1024]
['Activation', (None, 4, 4, 256), 0]
['Conv2D', (None, 4, 4, 256), 590080, 'same', 'linear', 'RandomUniform']
['BatchNormalization', (None, 4, 4, 256), 1024]
['Activation', (None, 4, 4, 256), 0]
['Conv2D', (None, 4, 4, 1024), 263168, 'valid', 'linear', 'RandomUniform']
['BatchNormalization', (None, 4, 4, 1024), 4096]
['Add', (None, 4, 4, 1024), 0]
['Activation', (None, 4, 4, 1024), 0]
['Conv2D', (None, 4, 4, 256), 262400, 'valid', 'linear', 'RandomUniform']
['BatchNormalization', (None, 4, 4, 256), 1024]
['Activation', (None, 4, 4, 256), 0]
['Conv2D', (None, 4, 4, 256), 590080, 'same', 'linear', 'RandomUniform']
['BatchNormalization', (None, 4, 4, 256), 1024]
['Activation', (None, 4, 4, 256), 0]
['Conv2D', (None, 4, 4, 1024), 263168, 'valid', 'linear', 'RandomUniform']
['BatchNormalization', (None, 4, 4, 1024), 4096]
['Add', (None, 4, 4, 1024), 0]
['Activation', (None, 4, 4, 1024), 0]
['Conv2D', (None, 4, 4, 256), 262400, 'valid', 'linear', 'RandomUniform']
['BatchNormalization', (None, 4, 4, 256), 1024]
['Activation', (None, 4, 4, 256), 0]
['Conv2D', (None, 4, 4, 256), 590080, 'same', 'linear', 'RandomUniform']
['BatchNormalization', (None, 4, 4, 256), 1024]
['Activation', (None, 4, 4, 256), 0]
['Conv2D', (None, 4, 4, 1024), 263168, 'valid', 'linear', 'RandomUniform']
['BatchNormalization', (None, 4, 4, 1024), 4096]
['Add', (None, 4, 4, 1024), 0]
['Activation', (None, 4, 4, 1024), 0]
['Conv2D', (None, 4, 4, 256), 262400, 'valid', 'linear', 'RandomUniform']
['BatchNormalization', (None, 4, 4, 256), 1024]
['Activation', (None, 4, 4, 256), 0]
['Conv2D', (None, 4, 4, 256), 590080, 'same', 'linear', 'RandomUniform']
['BatchNormalization', (None, 4, 4, 256), 1024]
['Activation', (None, 4, 4, 256), 0]
['Conv2D', (None, 4, 4, 1024), 263168, 'valid', 'linear', 'RandomUniform']
['BatchNormalization', (None, 4, 4, 1024), 4096]
['Add', (None, 4, 4, 1024), 0]
['Activation', (None, 4, 4, 1024), 0]
['Conv2D', (None, 4, 4, 256), 262400, 'valid', 'linear', 'RandomUniform']
['BatchNormalization', (None, 4, 4, 256), 1024]
['Activation', (None, 4, 4, 256), 0]
['Conv2D', (None, 4, 4, 256), 590080, 'same', 'linear', 'RandomUniform']
['BatchNormalization', (None, 4, 4, 256), 1024]
['Activation', (None, 4, 4, 256), 0]
['Conv2D', (None, 4, 4, 1024), 263168, 'valid', 'linear', 'RandomUniform']
['BatchNormalization', (None, 4, 4, 1024), 4096]
['Add', (None, 4, 4, 1024), 0]
['Activation', (None, 4, 4, 1024), 0]
['Conv2D', (None, 2, 2, 512), 524800, 'valid', 'linear', 'GlorotUniform']
['BatchNormalization', (None, 2, 2, 512), 2048]
['Activation', (None, 2, 2, 512), 0]
['Conv2D', (None, 2, 2, 512), 2359808, 'same', 'linear', 'GlorotUniform']
['BatchNormalization', (None, 2, 2, 512), 2048]
['Activation', (None, 2, 2, 512), 0]
['Conv2D', (None, 2, 2, 2048), 1050624, 'valid', 'linear', 'GlorotUniform']
['Conv2D', (None, 2, 2, 2048), 2099200, 'valid', 'linear', 'GlorotUniform']
['BatchNormalization', (None, 2, 2, 2048), 8192]
['BatchNormalization', (None, 2, 2, 2048), 8192]
['Add', (None, 2, 2, 2048), 0]
['Activation', (None, 2, 2, 2048), 0]
['Conv2D', (None, 2, 2, 512), 1049088, 'valid', 'linear', 'RandomUniform']
['BatchNormalization', (None, 2, 2, 512), 2048]
['Activation', (None, 2, 2, 512), 0]
['Conv2D', (None, 2, 2, 512), 2359808, 'same', 'linear', 'RandomUniform']
['BatchNormalization', (None, 2, 2, 512), 2048]
['Activation', (None, 2, 2, 512), 0]
['Conv2D', (None, 2, 2, 2048), 1050624, 'valid', 'linear', 'RandomUniform']
['BatchNormalization', (None, 2, 2, 2048), 8192]
['Add', (None, 2, 2, 2048), 0]
['Activation', (None, 2, 2, 2048), 0]
['Conv2D', (None, 2, 2, 512), 1049088, 'valid', 'linear', 'RandomUniform']
['BatchNormalization', (None, 2, 2, 512), 2048]
['Activation', (None, 2, 2, 512), 0]
['Conv2D', (None, 2, 2, 512), 2359808, 'same', 'linear', 'RandomUniform']
['BatchNormalization', (None, 2, 2, 512), 2048]
['Activation', (None, 2, 2, 512), 0]
['Conv2D', (None, 2, 2, 2048), 1050624, 'valid', 'linear', 'RandomUniform']
['BatchNormalization', (None, 2, 2, 2048), 8192]
['Add', (None, 2, 2, 2048), 0]
['Activation', (None, 2, 2, 2048), 0]
['AveragePooling2D', (None, 1, 1, 2048), 0]
['Flatten', (None, 2048), 0]
['Dense', (None, 6), 12294, 'softmax']
8

The formula for the number of parameters in a Conv2D layer is:

(f * f * nC_{in} + 1 ) * nC_{out}

The +1 there is for the bias term for each filter. For the layer that fails, we have:

nC_{in} = nC_{out} = 128

If you solve for f with both the expected and actual values, it becomes pretty clear what you need to look for. :nerd_face:

9

Hi Paul,

Thanks for the info, but I am still trying to figure this out. I have the exact same output for the model summary as you, with exception of three lines… 54, 64, and 74. The difference in the values here (147584 yours and 65664 mine) account for the difference of the parameters as a whole.

I was able to follow the code through the output of line 54, which seems to be coming from stage 3 call to convolutional_block which leads to the first Conv2D layer. The value of X shape before the Conv2D layer is (8,8,128) and f is 3. According to the equation given for number of parameters, and based on nCin being equal to nCout, we should have 147584… your result. Are you saying that my nCout is somehow wrong and where is that coming from?

Thanks,

Bob

10

What is the independent variable in that expression? It is f, right? Which is a parameter you specified.

Did you try what I suggested: solve the equation with each output value and see what values you get for f?

11

Sorry, when I composed my previous reply I was away from my computer and on my phone. It looks like you’ve narrowed it down to exactly the point of the error. And what you describe sounds right. Have you compared to the actual arguments you passed to convolutional_block in that instance? This whole assignment is a painstaking exercise in transcribing and proofreading.

12

No, my theory is that you are calling convolutional_block with f = 2.

(3 * 3 * 128 + 1) * 128 = 147584

(2 * 2 * 128 + 1) * 128 = 65664

It might also be worth a quick check to make sure that your notebook is in sync and WYSIWYG. Try this:

Kernel -> Restart and Clear Output
Save
Cell -> Run All

Then check and see if you still get the same test failure that you showed above.

The point of the above experiment is that if you type new code into a function cell and then just call the cell again, it runs the old code. You have to actually click “Shift-Enter” on the modified cell to get the new code incorporated into the runtime image. Or doing the “Kernel → Restart and Clear Output” is a sledgehammer way to get everything back in sync.

13

What you are saying makes sense, and I’ve added print statements inside of convolutional_block to print the arguments being passed.

in convolutional_block, X.shape=(None, 15, 15, 64), f=3, filters=[64, 64, 256], s=1
in convolutional_block, X.shape=(None, 15, 15, 256), f=3, filters=[128, 128, 512], s=2
in convolutional_block, X.shape=(None, 8, 8, 512), f=3, filters=[256, 256, 1024], s=2
in convolutional_block, X.shape=(None, 4, 4, 1024), f=3, filters=[512, 512, 2048], s=2

So I’ve verified f is always 3.

In convolutional_block, the only time f is being used as the value to kernel_size is in the “Second component of main path”, and I’ve verified in the call to Conv2d that kernel_size = f. I’ve restarted the kernel of the notebook and cleared the output, but I am still getting the same results.

14

Interesting. Ok, I’m out of theories, so I think it will require actually looking at your code to see if we can get to the next level. We can’t do that on a public thread, but please check your DMs for a message from me.

15

To close the loop on the public thread, we had the right theory that the f value should have been 3, but was specified as 2. The only problem was that we were looking for the error in the wrong place: it was on some of the calls to identity_block, not convolutional_block. Both of them call Conv2D …

2 Likes