C5 W1 music_inference_model

Course Q&A Deep Learning Specialization Sequence Models
1

I get this error and have no idea what is wrong with my code.
I already tried to restart the kernel and rerun all the cells without success.
Every hint or help is appreciated!

ValueError Traceback (most recent call last)
/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py in _apply_op_helper(op_type_name, name, **keywords)
469 as_ref=input_arg.is_ref,
→ 470 preferred_dtype=default_dtype)
471 except TypeError as err:

/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py in convert_to_tensor(value, dtype, name, as_ref, preferred_dtype, dtype_hint, ctx, accepted_result_types)
1474 "Tensor conversion requested dtype %s for Tensor with dtype %s: r"
→ 1475 (dtype.name, value.dtype.name, value))
1476 return value

ValueError: Tensor conversion requested dtype int32 for Tensor with dtype float32: <tf.Tensor ‘kernel:0’ shape=(90, 256) dtype=float32>

During handling of the above exception, another exception occurred:

TypeError Traceback (most recent call last)
in
----> 1 inference_model = music_inference_model(LSTM_cell, densor, Ty = 50)

in music_inference_model(LSTM_cell, densor, Ty)
38 for t in range(Ty):
39 # Step 2.A: Perform one step of LSTM_cell. Use “x”, not “x0” (≈1 line)
—> 40 a, _, c = LSTM_cell(inputs=x, initial_state = [a, c])
41
42 # Step 2.B: Apply Dense layer to the hidden state output of the LSTM_cell (≈1 line)

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/layers/recurrent.py in call(self, inputs, initial_state, constants, **kwargs)
707 # Perform the call with temporarily replaced input_spec
708 self.input_spec = full_input_spec
→ 709 output = super(RNN, self).call(full_input, **kwargs)
710 # Remove the additional_specs from input spec and keep the rest. It is
711 # important to keep since the input spec was populated by build(), and

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py in call(self, *args, **kwargs)
924 if _in_functional_construction_mode(self, inputs, args, kwargs, input_list):
925 return self._functional_construction_call(inputs, args, kwargs,
→ 926 input_list)
927
928 # Maintains info about the Layer.call stack.

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py in _functional_construction_call(self, inputs, args, kwargs, input_list)
1115 try:
1116 with ops.enable_auto_cast_variables(self._compute_dtype_object):
→ 1117 outputs = call_fn(cast_inputs, *args, **kwargs)
1118
1119 except errors.OperatorNotAllowedInGraphError as e:

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/layers/recurrent_v2.py in call(self, inputs, mask, training, initial_state)
1181 else:
1182 (last_output, outputs, new_h, new_c,
→ 1183 runtime) = lstm_with_backend_selection(**normal_lstm_kwargs)
1184
1185 states = [new_h, new_c]

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/layers/recurrent_v2.py in lstm_with_backend_selection(inputs, init_h, init_c, kernel, recurrent_kernel, bias, mask, time_major, go_backwards, sequence_lengths, zero_output_for_mask)
1556 # grappler will kick in during session execution to optimize the graph.
1557 last_output, outputs, new_h, new_c, runtime = defun_standard_lstm(
→ 1558 **params)
1559 function.register(defun_gpu_lstm, **params)
1560

/opt/conda/lib/python3.7/site-packages/tensorflow/python/eager/function.py in call(self, *args, **kwargs)
2826 “”“Calls a graph function specialized to the inputs.”""
2827 with self._lock:
→ 2828 graph_function, args, kwargs = self._maybe_define_function(args, kwargs)
2829 return graph_function._filtered_call(args, kwargs) # pylint: disable=protected-access
2830

/opt/conda/lib/python3.7/site-packages/tensorflow/python/eager/function.py in _maybe_define_function(self, args, kwargs)
3211
3212 self._function_cache.missed.add(call_context_key)
→ 3213 graph_function = self._create_graph_function(args, kwargs)
3214 self._function_cache.primary[cache_key] = graph_function
3215 return graph_function, args, kwargs

/opt/conda/lib/python3.7/site-packages/tensorflow/python/eager/function.py in _create_graph_function(self, args, kwargs, override_flat_arg_shapes)
3073 arg_names=arg_names,
3074 override_flat_arg_shapes=override_flat_arg_shapes,
→ 3075 capture_by_value=self._capture_by_value),
3076 self._function_attributes,
3077 function_spec=self.function_spec,

/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py in func_graph_from_py_func(name, python_func, args, kwargs, signature, func_graph, autograph, autograph_options, add_control_dependencies, arg_names, op_return_value, collections, capture_by_value, override_flat_arg_shapes)
984 _, original_func = tf_decorator.unwrap(python_func)
985
→ 986 func_outputs = python_func(*func_args, **func_kwargs)
987
988 # invariant: func_outputs contains only Tensors, CompositeTensors,

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/layers/recurrent_v2.py in standard_lstm(inputs, init_h, init_c, kernel, recurrent_kernel, bias, mask, time_major, go_backwards, sequence_lengths, zero_output_for_mask)
1313 input_length=(sequence_lengths
1314 if sequence_lengths is not None else timesteps),
→ 1315 zero_output_for_mask=zero_output_for_mask)
1316 return (last_output, outputs, new_states[0], new_states[1],
1317 _runtime(_RUNTIME_CPU))

/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py in wrapper(*args, **kwargs)
199 “”“Call target, and fall back on dispatchers if there is a TypeError.”""
200 try:
→ 201 return target(*args, **kwargs)
202 except (TypeError, ValueError):
203 # Note: convert_to_eager_tensor currently raises a ValueError, not a

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/backend.py in rnn(step_function, inputs, initial_states, go_backwards, mask, constants, unroll, input_length, time_major, zero_output_for_mask)
4212 # the value is discarded.
4213 output_time_zero, _ = step_function(
→ 4214 input_time_zero, tuple(initial_states) + tuple(constants))
4215 output_ta = tuple(
4216 tensor_array_ops.TensorArray(

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/layers/recurrent_v2.py in step(cell_inputs, cell_states)
1289 c_tm1 = cell_states[1] # previous carry state
1290
→ 1291 z = K.dot(cell_inputs, kernel)
1292 z += K.dot(h_tm1, recurrent_kernel)
1293 z = K.bias_add(z, bias)

/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py in wrapper(*args, **kwargs)
199 “”“Call target, and fall back on dispatchers if there is a TypeError.”""
200 try:
→ 201 return target(*args, **kwargs)
202 except (TypeError, ValueError):
203 # Note: convert_to_eager_tensor currently raises a ValueError, not a

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/backend.py in dot(x, y)
1829 out = sparse_ops.sparse_tensor_dense_matmul(x, y)
1830 else:
→ 1831 out = math_ops.matmul(x, y)
1832 return out
1833

/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py in wrapper(*args, **kwargs)
199 “”“Call target, and fall back on dispatchers if there is a TypeError.”""
200 try:
→ 201 return target(*args, **kwargs)
202 except (TypeError, ValueError):
203 # Note: convert_to_eager_tensor currently raises a ValueError, not a

/opt/conda/lib/python3.7/site-packages/tensorflow/python/ops/math_ops.py in matmul(a, b, transpose_a, transpose_b, adjoint_a, adjoint_b, a_is_sparse, b_is_sparse, name)
3253 else:
3254 return gen_math_ops.mat_mul(
→ 3255 a, b, transpose_a=transpose_a, transpose_b=transpose_b, name=name)
3256
3257

/opt/conda/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py in mat_mul(a, b, transpose_a, transpose_b, name)
5640 _, _, _op, _outputs = _op_def_library._apply_op_helper(
5641 “MatMul”, a=a, b=b, transpose_a=transpose_a, transpose_b=transpose_b,
→ 5642 name=name)
5643 _result = _outputs[:]
5644 if _execute.must_record_gradient():

/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py in _apply_op_helper(op_type_name, name, **keywords)
504 "%s type %s of argument ’s'."
505 (prefix, dtypes.as_dtype(attrs[input_arg.type_attr]).name,
→ 506 inferred_from[input_arg.type_attr]))
507
508 types = [values.dtype]

TypeError: Input ‘b’ of ‘MatMul’ Op has type float32 that does not match type int32 of argument ‘a’.

2

If you decode all of those asserts, it means that this line of code in your step() function:
z = K.dot(cell_inputs, kernel)
is throwing an error because the two operands aren’t the same size.

3

I’m sorry, but this does not help me at all…

4

It is difficult to point out the exact location to be fixed from this trace, but I try to list up some potential hints.

It looks like the state in LSTM cell is being incorrectly updated. That’s an observation.

The number of LSTM is initially set by

LSTM_cell = LSTM(n_a, return_state = True) # Used in Step 2.C

At this time, n_a is a global variable, and is 64.
After that, all dimensions for hidden state and cell state are set by retrieving the number of units from LSTM_cell.

n_a = LSTM_cell.units

Then, in music_inference_model() (and, of course, djmodel()), state values a and c are updated. If state values of a and c that you get as a return of LSTM_cell() are used as inputs to next LSTM_cell() call with no modifications, then, I think LSTM kernel is kept safe. But, in this iteration, the shape of a or c is incorrectly updated and fed into LSTM_cell() at the next iteration, then, this types of error may occur.
So, the first thing to check is that the shape of state values, a and c, are untouched.

5

I added some print statements in the code.
The shapes of a and c seem to be untouched.

The loop runs exactly once, beginning with the second iteration it fails:

a before: (None, 64)
c before: (None, 64)
a after: (None, 64)
c after: (None, 64)
a before: (None, 64)
c before: (None, 64)

ValueError Traceback (most recent call last)
/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py in _apply_op_helper(op_type_name, name, **keywords)
469 as_ref=input_arg.is_ref,
→ 470 preferred_dtype=default_dtype)
471 except TypeError as err:

/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py in convert_to_tensor(value, dtype, name, as_ref, preferred_dtype, dtype_hint, ctx, accepted_result_types)
1474 "Tensor conversion requested dtype %s for Tensor with dtype %s: r"
→ 1475 (dtype.name, value.dtype.name, value))
1476 return value
…

6

OK. understood.

Here is my weights for LSTM_cell. (you can get by LSTM_cell.weights). Those are

[<tf.Variable ‘lstm_11/lstm_cell_11/kernel:0’ shape=(90, 256) dtype=float32, numpy=
array([[ 0.23384853, 0.77213407, 0.4358826 , …, 0.0408756 ,
-0.64276934, 0.4222921 ],
…
-1.0703363 , 0.31678507]], dtype=float32)>,
<tf.Variable ‘lstm_11/lstm_cell_11/recurrent_kernel:0’ shape=(64, 256) dtype=float32, numpy=
array([[ 0.14301907, -0.11659534, 0.03532916, …, 0.4620593 ,
0.26161996, 0.40132025],
…
0.02005301, 0.18978874]], dtype=float32)>,
<tf.Variable ‘lstm_11/lstm_cell_11/bias:0’ shape=(256,) dtype=float32, numpy=
array([ 0.5073488 , 0.61192894, 0.64600945, 0.66372526, 0.46812612,
…
0.7437702 ], dtype=float32)>]

Next candidate is, of course, . x. I suppose there is a dot product of x and kernel (90,256) in Keras LSTM code. Let’s check the shape of x is maintained correctly. It needs to be (None, 90) after on-hot encoding and (None, 1, 90) after RepeatVector().

7

Sorry, I did not realize that the “step()” function is buried in the Keras implementation.
In the call to LSTM_cell(…), try removing “inputs=”, so you’re just passing ‘x’.

8

I changed the call to LSTM_cell to

a, _, c = LSTM_cell(x, initial_state = [a, c])

Then I did a kernel → Restart & run all.

I get exactly the same error.

9

Hi Nobu_Asi,
I had a look to the shape of x. At least to my understanding the shape should be ok:

shape x: (None, 1, 90)
shape x after argmax: (None,)
shape x after onehot: (None, 90)
shape x after RepeatVector: (None, 1, 90)

Do you agree?

10

Looks OK. This is curious, since as log as shapes for a, c, and x are correct, LSTM_cell should generate some values regardless of those values.
You can DM your notebook to me.

Nobu

11

By the way, another possibility is to set x tensor to int32 type.
Since a kernel in LSTM is “shape=(90, 256) dtype=float32”, python/tensorflow tries to convert x, and causes the following error.

ValueError: Tensor conversion requested dtype int32 for Tensor with dtype float32: <tf.Tensor ‘kernel:0’ shape=(90, 256) dtype=float32>

I believe that, in default, the output from tf.one_hot() is tf.float32. But, if you specify “dtype=int32”, of course, the output is tf.int32, which causes the above error.

Of course, the above is one of possibilities. But, it may be worth to check the data type of x at the 2nd iteration.

1 Like
12

Oh man…
my fault was that I wrote

    x = tf.one_hot(x, depth=n_values, on_value = 1)

instead of

    x = tf.one_hot(x, depth=n_values)

The ‘1’ for on_value is interpreted as tf.int32, I should have written ‘1.0’
I wasted nearly a day to figure this out, but finally it works now…
Thank you guys for helping me out, I appreciate it a lot!