C3W3 - Training the Model - Exercise 3

Course Q&A Natural Language Processing NLP with Sequence Models
1

Hi!

I am currently struggling for more than 3 hours in the same exercise.

I can’t compile my final model, although my unit test for this exercise passes.

I get the following error right after the output “Epoch 01”:

Epoch 1/2
---------------------------------------------------------------------------
UnimplementedError                        Traceback (most recent call last)
Cell In[23], line 9
      3 train_generator = train_dataset.shuffle(len(train_Q1),
      4                                         seed=7, 
      5                                         reshuffle_each_iteration=True).batch(batch_size=batch_size)
      6 val_generator = val_dataset.shuffle(len(val_Q1), 
      7                                    seed=7,
      8                                    reshuffle_each_iteration=True).batch(batch_size=batch_size)
----> 9 model = train_model(Siamese, TripletLoss,text_vectorization, 
     10                                             train_generator, 
     11                                             val_generator, 
     12                                             train_steps=train_steps,)

Cell In[22], line 29, in train_model(Siamese, TripletLoss, text_vectorizer, train_dataset, val_dataset, d_feature, lr, train_steps)
     25 model.compile(loss=TripletLoss,
     26               optimizer = tf.keras.optimizers.Adam(lr)
     27         )
     28 # Train the model 
---> 29 model.fit(train_dataset,
     30           epochs = train_steps,
     31           validation_data = val_dataset
     32          )
     34 ### END CODE HERE ###
     36 return model

File /usr/local/lib/python3.8/dist-packages/keras/src/utils/traceback_utils.py:70, in filter_traceback.<locals>.error_handler(*args, **kwargs)
     67     filtered_tb = _process_traceback_frames(e.__traceback__)
     68     # To get the full stack trace, call:
     69     # `tf.debugging.disable_traceback_filtering()`
---> 70     raise e.with_traceback(filtered_tb) from None
     71 finally:
     72     del filtered_tb

File /usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/execute.py:53, in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
     51 try:
     52   ctx.ensure_initialized()
---> 53   tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
     54                                       inputs, attrs, num_outputs)
     55 except core._NotOkStatusException as e:
     56   if name is not None:

UnimplementedError: Graph execution error:

Detected at node 'SiameseModel/Cast' defined at (most recent call last):
    File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
      return _run_code(code, main_globals, None,
    File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
      exec(code, run_globals)
    File "/usr/local/lib/python3.8/dist-packages/ipykernel_launcher.py", line 17, in <module>
      app.launch_new_instance()
    File "/usr/local/lib/python3.8/dist-packages/traitlets/config/application.py", line 1043, in launch_instance
      app.start()
    File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelapp.py", line 728, in start
      self.io_loop.start()
    File "/usr/local/lib/python3.8/dist-packages/tornado/platform/asyncio.py", line 195, in start
      self.asyncio_loop.run_forever()
    File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
      self._run_once()
    File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once
      handle._run()
    File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run
      self._context.run(self._callback, *self._args)
    File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 516, in dispatch_queue
      await self.process_one()
    File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 505, in process_one
      await dispatch(*args)
    File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 412, in dispatch_shell
      await result
    File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 740, in execute_request
      reply_content = await reply_content
    File "/usr/local/lib/python3.8/dist-packages/ipykernel/ipkernel.py", line 422, in do_execute
      res = shell.run_cell(
    File "/usr/local/lib/python3.8/dist-packages/ipykernel/zmqshell.py", line 540, in run_cell
      return super().run_cell(*args, **kwargs)
    File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3009, in run_cell
      result = self._run_cell(
    File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3064, in _run_cell
      result = runner(coro)
    File "/usr/local/lib/python3.8/dist-packages/IPython/core/async_helpers.py", line 129, in _pseudo_sync_runner
      coro.send(None)
    File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3269, in run_cell_async
      has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
    File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3448, in run_ast_nodes
      if await self.run_code(code, result, async_=asy):
    File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3508, in run_code
      exec(code_obj, self.user_global_ns, self.user_ns)
    File "/tmp/ipykernel_6661/1427930616.py", line 9, in <module>
      model = train_model(Siamese, TripletLoss,text_vectorization,
    File "/tmp/ipykernel_6661/687534374.py", line 29, in train_model
      model.fit(train_dataset,
    File "/usr/local/lib/python3.8/dist-packages/keras/src/utils/traceback_utils.py", line 65, in error_handler
      return fn(*args, **kwargs)
    File "/usr/local/lib/python3.8/dist-packages/keras/src/engine/training.py", line 1742, in fit
      tmp_logs = self.train_function(iterator)
    File "/usr/local/lib/python3.8/dist-packages/keras/src/engine/training.py", line 1338, in train_function
      return step_function(self, iterator)
    File "/usr/local/lib/python3.8/dist-packages/keras/src/engine/training.py", line 1322, in step_function
      outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "/usr/local/lib/python3.8/dist-packages/keras/src/engine/training.py", line 1303, in run_step
      outputs = model.train_step(data)
    File "/usr/local/lib/python3.8/dist-packages/keras/src/engine/training.py", line 1080, in train_step
      y_pred = self(x, training=True)
    File "/usr/local/lib/python3.8/dist-packages/keras/src/utils/traceback_utils.py", line 65, in error_handler
      return fn(*args, **kwargs)
    File "/usr/local/lib/python3.8/dist-packages/keras/src/engine/training.py", line 569, in __call__
      return super().__call__(*args, **kwargs)
    File "/usr/local/lib/python3.8/dist-packages/keras/src/utils/traceback_utils.py", line 65, in error_handler
      return fn(*args, **kwargs)
    File "/usr/local/lib/python3.8/dist-packages/keras/src/engine/base_layer.py", line 1150, in __call__
      outputs = call_fn(inputs, *args, **kwargs)
    File "/usr/local/lib/python3.8/dist-packages/keras/src/utils/traceback_utils.py", line 96, in error_handler
      return fn(*args, **kwargs)
    File "/usr/local/lib/python3.8/dist-packages/keras/src/engine/functional.py", line 512, in call
      return self._run_internal_graph(inputs, training=training, mask=mask)
    File "/usr/local/lib/python3.8/dist-packages/keras/src/engine/functional.py", line 651, in _run_internal_graph
      y = self._conform_to_reference_input(y, ref_input=x)
    File "/usr/local/lib/python3.8/dist-packages/keras/src/engine/functional.py", line 748, in _conform_to_reference_input
      tensor = tf.cast(tensor, dtype=ref_input.dtype)
Node: 'SiameseModel/Cast'
2 root error(s) found.
  (0) UNIMPLEMENTED:  Cast string to float is not supported
	 [[{{node SiameseModel/Cast}}]]
  (1) CANCELLED:  Function was cancelled before it was started
0 successful operations.
0 derived errors ignored. [Op:__inference_train_function_26285]

Also, when evaluating (accuracy), I can’t seem to make it work.

I would greatly appreciate if anyone can help me!

Thank you!!

2

Hi @Andre_Jose_de_Queiro

Please check this thread and see if it helps. In particular, in Exercise 01 - make sure you defined inputs of type tf.string.

Cheers

2 Likes
3

Thank you VERY much. It solved the problem!

4

Hi, Mentor!

I revised the type as you suggested. I passed all the tests for this exercise. However, I still got a warning when I ran the following cell

Test your function!

w3_unittest.test_train_model(train_model, Siamese, TripletLoss)

I pasted the warning below. I don’t understand what it means. What should I revise then? Any advices? Many thanks!

Sincerely,
Shining

WARNING:tensorflow:5 out of the last 6972 calls to <function PreprocessingLayer.make_adapt_function..adapt_step at 0x7f3af01c4790> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to Lepsza wydajność dzięki funkcji tf.  |  TensorFlow Core and tf.function  |  TensorFlow v2.16.1 for more details.
All tests passed!

5

it is just warning log information from tensorflow, you can ignore it.

6

Thank you for your explanation!