ResNets outside the notebook

Course Q&A Deep Learning Specialization Convolutional Neural Networks
1

I am trying to download the code from the notebooks to my computer and execute them there but I have some problems, all notebooks work fine until I tried the ResNet one, I found a problem where it said something about training parameter for BatchNormalization, i fixed that one by doing

X = BatchNormalization(axis = 3)(X, training=training)

instead of

X = BatchNormalization(axis = 3)(X)

Now I found another problem in convolutional_block() that says:

Cannot convert ‘2’ to a shape.

Specifically in here:

Cell In[30], line 27
     23 X3 = np.ones((1, 4, 4, 3)) * 3
     25 X = np.concatenate((X1, X2, X3), axis=0).astype(np.float32)
---> 27 A = target(X, f=2, filters=[2, 4, 6], s=4, training=False, initializer=tf.keras.initializers.RandomUniform())
     28 assert A.shape == (3, 1, 1, 6), "Wrong shape. Make sure you are using the stride values as expected."
     29 assert np.allclose(A.numpy(), convolutional_block_output1), "Wrong values when training=False."

Cell In[29], line 27
     23 ##### MAIN PATH #####
     24 
     25 # First component of main path glorot_uniform(seed=0)
     26 X = Conv2D(filters = 2, kernel_size = (1,1), strides = (4, 4), padding='valid')(X)
---> 27 X = BatchNormalization(axis = 3)(X, training=training)
     28 X = Activation('relu')(X)
     30 ## Second component of main path (≈3 lines)

I think it has something to do with the tensorflow version but I would like to use the last one, any ideas on what the problem is? Following the documentation (tf.keras.layers.BatchNormalization  |  TensorFlow v2.16.1) seems like it is ok to me.

2

Hello @Alejandro_de_la_Cruz,

Just from this message, maybe you were passing an integer as shape while it required a Tuple. If you did something like input_shape=2, try input_shape=(2, ) instead if you are sure that (2, ) is the right shape.

From the message you have shared, it is unclear where the problem is. Perhaps you could share the whole error message?

Or, could you identify what that '2' is? Where did you put it?

Cheers,
Raymond

3

The full error is this:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[12], line 38
     34     assert np.allclose(B.numpy(), convolutional_block_output2), "Wrong values when training=True."
     36     print('\033[92mAll tests passed!')
---> 38 test(convolutional_block)

Cell In[12], line 27
     23 X3 = np.ones((1, 4, 4, 3)) * 3
     25 X = np.concatenate((X1, X2, X3), axis=0).astype(np.float32)
---> 27 A = target(X, f=2, filters=[2, 4, 6], s=4, training=False, initializer=tf.keras.initializers.RandomUniform())
     28 assert A.shape == (3, 1, 1, 6), "Wrong shape. Make sure you are using the stride values as expected."
     29 assert np.allclose(A.numpy(), convolutional_block_output1), "Wrong values when training=False."

Cell In[11], line 27
     23 ##### MAIN PATH #####
     24 
     25 # First component of main path glorot_uniform(seed=0)
     26 X = Conv2D(filters = 2, kernel_size = (1,1), strides = (4, 4), padding='valid')(X)
---> 27 X = BatchNormalization(axis = 3)(X, training=training)
     28 X = Activation('relu')(X)
     30 ## Second component of main path (≈3 lines)

File ~\AppData\Roaming\Python\Python310\site-packages\keras\src\utils\traceback_utils.py:122, in filter_traceback.<locals>.error_handler(*args, **kwargs)
    119     filtered_tb = _process_traceback_frames(e.__traceback__)
    120     # To get the full stack trace, call:
    121     # `keras.config.disable_traceback_filtering()`
--> 122     raise e.with_traceback(filtered_tb) from None
    123 finally:
    124     del filtered_tb

File d:\11ResNet\resnets_utils.py:23, in BatchNormalization.build(self, input_shape)
     22 def build(self, input_shape):
---> 23     self.beta = self.add_weight(
     24         name="beta",
     25         shape=(input_shape[self.axis]),
     26         initializer="zeros",
     27         trainable=True,
     28     )
     30     self.gamma = self.add_weight(
     31         name="gamma",
     32         shape=(input_shape[self.axis]),
     33         initializer="ones",
     34         trainable=True,
     35     )
     37     self.moving_mean = self.add_weight(
     38         name="moving_mean",
     39         shape=(input_shape[self.axis]),
     40         initializer=tf.initializers.zeros,
     41         trainable=False)

ValueError: Cannot convert '2' to a shape.

And that 2 refers to the number of filters in Conv2D, because if I change it to 4 then the error says 4 instead of 2.

X = Conv2D(filters = 2, kernel_size = (1,1), strides = (4, 4), padding=‘valid’)(X)

If i put filters = (2,) then I get another error ‘TypeError: ‘<=’ not supported between instances of ‘tuple’ and ‘int’’. I also checked the documentation for Conv2D (tf.keras.layers.Conv2D  |  TensorFlow v2.16.1) and again I do not see anything wrong.

Also, the code works on the notebooks from coursera, I only have the when I download it, thats why I thought the problem may be in the tensorflow version.

4

Hello @Alejandro_de_la_Cruz,

If we follow the arrows along the error traceback, it actually pointed to BatchNormalization, so it should have nothing to do with the 2 for Conv’s filter size. Please use 2 for filters.

image

The code in the traceback also shows that it used an integer for the shape parameter instead of a tuple which is the standard for today’s tensorflow.

image

I would suspect the tensorflow version too.

Cheers,
Raymond

5

@Alejandro_de_la_Cruz,

The error traceback also tells us that the BatchNormalization object was defined in a custom script instead of from a tensorflow package.
image

In today’s tensorflow, their BatchNormalization will convert the integer into a tuple to avoid that error.

Maybe (not verified yet) the tensorflow version used on Cousera would automatically correct this but today’s tensorflow wouldn’t.

In other words, if you use your tensorflow’s batchnormalization instead of the script’s, that problem may go away (but new problem in subsequent code may arise for using a different implementation). Otherwise, you may want to just build an enviornment like Cousera’s.

Cheers,
Raymond

1 Like
6

Hello @Alejandro_de_la_Cruz,

I had same issue due to my local TensorFlow version not matching the one used in the Lab. Like you did, I attempted to adapt my code to incompatible environment like adding training=True as 2.16 version requires but no way.

You can find one part of the solution here to set a special aligned virtual environment.

I prepare another post to fix the other issue(s) that came up since fixing that one.

Regards,

Francis