Course 4 week 1 happy model build

I’m getting an error when running the code for the assignament 2:

happy_model.summary()
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-37-46aed12cf67b> in <module>
----> 1 happy_model.summary()

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py in summary(self, line_length, positions, print_fn)
   2349     """
   2350     if not self.built:
-> 2351       raise ValueError('This model has not yet been built. '
   2352                        'Build the model first by calling `build()` or calling '
   2353                        '`fit()` with some data, or specify '

ValueError: This model has not yet been built. Build the model first by calling `build()` or calling `fit()` with some data, or specify an `input_shape` argument in the first layer(s) for automatic build.

I’m having trouble understanding how the happy model is supposed to be built.

the formula that we’re given to update very clearly shows that we should be adding the layers as individual members of the list array inside of the tf.keras.Sequential() function.

def happyModel():
    input_shape = (1, 64,64,1)
    x = np.arange(np.prod(input_shape)).reshape(input_shape)
    model = tf.keras.Sequential([
            ## ZeroPadding2D with padding 3, input shape of 64 x 64 x 3

            ## Conv2D with 32 7x7 filters and stride of 1

            ## BatchNormalization for axis 3

            ## ReLU

            ## Max Pooling 2D with default parameters

            ## Flatten layer

            ## Dense layer with 1 unit for output & 'sigmoid' activation

            # YOUR CODE STARTS HERE
            
            
            # YOUR CODE ENDS HERE
    ])

but if I do that I get this error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-70-f33284fd82fe> in <module>
      1 happy_model = happyModel()
      2 # Print a summary for each layer
----> 3 for layer in summary(happy_model):
      4     print(layer)
      5 

~/work/release/W1A2/test_utils.py in summary(model)
     30     result = []
     31     for layer in model.layers:
---> 32         descriptors = [layer.__class__.__name__, layer.output_shape, layer.count_params()]
     33         if (type(layer) == Conv2D):
     34             descriptors.append(layer.padding)

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py in output_shape(self)
   2177     """
   2178     if not self._inbound_nodes:
-> 2179       raise AttributeError('The layer has never been called '
   2180                            'and thus has no defined output shape.')
   2181     all_output_shapes = set(

AttributeError: The layer has never been called and thus has no defined output shape.

the dense line that I added was this: layers.Dense(1) with the activation as sigmoid.

Now if I change this code instead to this:

model = tf.keras.models.Sequential()    
## Conv2D with 32 7x7 filters and stride of 1
function(),
## BatchNormalization for axis 3
function(),
    ## ReLU
function(),
    ## Max Pooling 2D with default parameters
function(),
    ## Flatten layer
function(),
    ## Dense layer with 1 unit for output & 'sigmoid' activation
layers.Dense(1)` with the activation as sigmoid

now this does work and I’m able to pass the first model. but then the problem is when trying to run this line
happy_model.fit(X_train, Y_train, epochs=10, batch_size=16)
I get this error:

Epoch 1/10
---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-75-a5c10c720821> in <module>
----> 1 happy_model.fit(X_train, Y_train, epochs=10, batch_size=16)

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py in _method_wrapper(self, *args, **kwargs)
    106   def _method_wrapper(self, *args, **kwargs):
    107     if not self._in_multi_worker_mode():  # pylint: disable=protected-access
--> 108       return method(self, *args, **kwargs)
    109 
    110     # Running inside `run_distribute_coordinator` already.

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing)
   1096                 batch_size=batch_size):
   1097               callbacks.on_train_batch_begin(step)
-> 1098               tmp_logs = train_function(iterator)
   1099               if data_handler.should_sync:
   1100                 context.async_wait()

/opt/conda/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py in __call__(self, *args, **kwds)
    778       else:
    779         compiler = "nonXla"
--> 780         result = self._call(*args, **kwds)
    781 
    782       new_tracing_count = self._get_tracing_count()

/opt/conda/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py in _call(self, *args, **kwds)
    838         # Lifting succeeded, so variables are initialized and we can run the
    839         # stateless function.
--> 840         return self._stateless_fn(*args, **kwds)
    841     else:
    842       canon_args, canon_kwds = \

/opt/conda/lib/python3.7/site-packages/tensorflow/python/eager/function.py in __call__(self, *args, **kwargs)
   2827     with self._lock:
   2828       graph_function, args, kwargs = self._maybe_define_function(args, kwargs)
-> 2829     return graph_function._filtered_call(args, kwargs)  # pylint: disable=protected-access
   2830 
   2831   @property

/opt/conda/lib/python3.7/site-packages/tensorflow/python/eager/function.py in _filtered_call(self, args, kwargs, cancellation_manager)
   1846                            resource_variable_ops.BaseResourceVariable))],
   1847         captured_inputs=self.captured_inputs,
-> 1848         cancellation_manager=cancellation_manager)
   1849 
   1850   def _call_flat(self, args, captured_inputs, cancellation_manager=None):

/opt/conda/lib/python3.7/site-packages/tensorflow/python/eager/function.py in _call_flat(self, args, captured_inputs, cancellation_manager)
   1922       # No tape is watching; skip to running the function.
   1923       return self._build_call_outputs(self._inference_function.call(
-> 1924           ctx, args, cancellation_manager=cancellation_manager))
   1925     forward_backward = self._select_forward_and_backward_functions(
   1926         args,

/opt/conda/lib/python3.7/site-packages/tensorflow/python/eager/function.py in call(self, ctx, args, cancellation_manager)
    548               inputs=args,
    549               attrs=attrs,
--> 550               ctx=ctx)
    551         else:
    552           outputs = execute.execute_with_cancellation(

/opt/conda/lib/python3.7/site-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
     58     ctx.ensure_initialized()
     59     tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
---> 60                                         inputs, attrs, num_outputs)
     61   except core._NotOkStatusException as e:
     62     if name is not None:

InvalidArgumentError:  Incompatible shapes: [16,64,64] vs. [16,1]
	 [[node Equal (defined at <ipython-input-75-a5c10c720821>:1) ]] [Op:__inference_train_function_2028]

Function call stack:
train_function

and I have no idea how to fix this.

I think another issue is that the jupyter notebook is calling for the model summary before it has been built, so it will always show the first error.

happy_model.summary()

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-87-46aed12cf67b> in <module>
----> 1 happy_model.summary()

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py in summary(self, line_length, positions, print_fn)
   2349     """
   2350     if not self.built:
-> 2351       raise ValueError('This model has not yet been built. '
   2352                        'Build the model first by calling `build()` or calling '
   2353                        '`fit()` with some data, or specify '

ValueError: This model has not yet been built. Build the model first by calling `build()` or calling `fit()` with some data, or specify an `input_shape` argument in the first layer(s) for automatic build.

from the keras website: The Sequential model

model = keras.Sequential(
    [
        layers.Dense(2, activation="relu"),
        layers.Dense(3, activation="relu"),
        layers.Dense(4),
    ]
)  # No weights at this stage!

# At this point, you can't do this:
# model.weights

# You also can't do this:
# model.summary()

# Call the model on a test input
x = tf.ones((1, 4))
y = model(x)
print("Number of weights after calling the model:", len(model.weights))  # 6

model.summary() cannot be called before adding something to the model.

This problem also affects the grader.

This assignment uses the Sequential model.

The sequential model is simply a python list of all of the layers you want the model to include.
You don’t apply any data at this point, that’s handled when you actually call the model.

So you define the list of layers between the square brackets [ ] ,with each layer separated by commas (as is used for any python list).

See the assignment instructions for creating the layers.

Later on, you will run one of the notebook cells that applies some data to the model you’re created.

1 Like

the error that I’m getting from the grader is this:

[ValidateApp | INFO] Validating '/home/jovyan/work/submitted/courseraLearner/W1A2/Convolution_model_Application.ipynb'
[ValidateApp | INFO] Executing notebook with kernel: python3
2021-09-12 05:41:13.927361: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2021-09-12 05:41:13.927398: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2021-09-12 05:41:15.147948: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2021-09-12 05:41:15.147980: W tensorflow/stream_executor/cuda/cuda_driver.cc:312] failed call to cuInit: UNKNOWN ERROR (303)
2021-09-12 05:41:15.148004: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (ip-10-2-101-91.ec2.internal): /proc/driver/nvidia/version does not exist
2021-09-12 05:41:15.148180: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-09-12 05:41:15.174700: I tensorflow/core/platform/profile_utils/cpu_utils.cc:104] CPU Frequency: 3000000000 Hz
2021-09-12 05:41:15.176532: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55a971b878c0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2021-09-12 05:41:15.176558: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
[ValidateApp | ERROR] Timeout waiting for execute reply (30s).
[ValidateApp | ERROR] Interrupting kernel
[ValidateApp | ERROR] Timeout waiting for execute reply (30s).
[ValidateApp | ERROR] Interrupting kernel
Tests failed on 1 cell(s)! These tests could be hidden. Please check your submission.
==========================================================================================
The following cell failed:

    happy_model = happyModel()
    # Print a summary for each layer
    for layer in summary(happy_model):
        print(layer)
        
    output = [['ZeroPadding2D', (None, 70, 70, 3), 0, ((3, 3), (3, 3))],
                ['Conv2D', (None, 64, 64, 32), 4736, 'valid', 'linear', 'GlorotUniform'...
                ['BatchNormalization', (None, 64, 64, 32), 128],
                ['ReLU', (None, 64, 64, 32), 0],
                ['MaxPooling2D', (None, 32, 32, 32), 0, (2, 2), (2, 2), 'valid'],
                ['Flatten', (None, 32768), 0],
                ['Dense', (None, 1), 32769, 'sigmoid']]
        
    comparator(summary(happy_model), output)

The error was:

    ---------------------------------------------------------------------------
    AttributeError                            Traceback (most recent call last)
    <ipython-input-8-f33284fd82fe> in <module>
          1 happy_model = happyModel()
          2 # Print a summary for each layer
    ----> 3 for layer in summary(happy_model):
          4     print(layer)
          5 
    
    ~/work/submitted/courseraLearner/W1A2/test_utils.py in summary(model)
         30     result = []
         31     for layer in model.layers:
    ---> 32         descriptors = [layer.__class__.__name__, layer.output_shape, layer....
         33         if (type(layer) == Conv2D):
         34             descriptors.append(layer.padding)
    
    /opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py...
       2177     """
       2178     if not self._inbound_nodes:
    -> 2179       raise AttributeError('The layer has never been called '
       2180                            'and thus has no defined output shape.')
       2181     all_output_shapes = set(
    
    AttributeError: The layer has never been called and thus has no defined output shap...

what it doesn’t like is the for layer in summary(happy_model) and the reason why it doesn’t like it is because it hasn’t been initialized with data.

If I run the code under 3.2
happy_model.fit(X_train, Y_train, epochs=10, batch_size=16)
and then go back to running the summary, it works.

I can post my code in here so that you can verify that it works except for how the summary is being called before any data is sent to it.

Note that you should only modify the notebook cells inside the functions where it says for you to add your code.

If you modified any of the other cells, or if you added any new cells, it’s going to cause endless problems.

I found the same problem as aari described.
I found that if we add 1 input layer before the ZeroPadding2D layer using keras Input.
That solved the problem of Input and we don’t have to use ‘add’ functions outside of the sequential constructor. Apparently, the instruction did not mention this input layer.

You do not need to add an input layer before ZeroPadding2D().

You mentioned we should not be adding a layer (response to @Thai_Giang_Dang), but this is how the keras documentation implements it (example @aari provided).

In the ZeroPadding2D documentation (first example) I saw this syntax, although it seems like it is just another way of adding a layer before the first layer:

input_shape = (1, 1, 2, 2)
x = np.arange(np.prod(input_shape)).reshape(input_shape)
y = tf.keras.layers.ZeroPadding2D(padding=1)(x)

Could you dive into how the input shape should be specified?

There is more than one way to specify the input shape.

Also, the course materials rarely follow the Keras documentation or examples.

Is instantiating a tensor at the beginning the best way to specify an input size within a sequential model? I am not sure what the best practice is

I don’t think there is a best practice. There are several equally-good practices.

For any of the layer functions which are subclasses of Layer, you can simply add the input_shape named parameter. So you don’t need a separate layer in order to specify the input shape. Just add that parameter to the first of the layers in your Sequential model.

Thank you for the guidance, I really appreciate it. I did not think of looking into what parameters the Layer object could accept - I was digging into the source code and found the other parameters that can be used: tensorflow/base_layer.py at 16ae544750975296dcadd1ebb8eb52190f17ab37 · tensorflow/tensorflow · GitHub

    # These properties should be set by the user via keyword arguments.
    # note that 'dtype', 'input_shape' and 'batch_input_shape'
    # are only applicable to input layers: do not pass these keywords
    # to non-input layers.
    allowed_kwargs = {
        'input_dim',
        'input_shape',
        'batch_input_shape',
        'batch_size',
        'weights',
        'activity_regularizer',
        'autocast',
        'implementation',
    }

I am sure this will come in handy in the future. Thanks again!