DLS course 4 week 4 exercise 6 train_step

Course Q&A Deep Learning Specialization Convolutional Neural Networks
1

I’m receiving this error about converting a tf tensor to a np array:

NotImplementedError Traceback (most recent call last)
Input In [44], in <cell line: 7>()
1 ### you cannot edit this cell
2
3 # You always must run the last cell before this one. You will get an error if not.
5 generated_image = tf.Variable(generated_image)
----> 7 train_step_test(train_step, generated_image)

File /tf/W4A2/public_tests.py:86, in train_step_test(target, generated_image)
82 def train_step_test(target, generated_image):
83 generated_image = tf.Variable(generated_image)
—> 86 J1 = target(generated_image)
87 print(J1)
88 assert type(J1) == EagerTensor, f"Wrong type {type(J1)} != {EagerTensor}"

File /usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py:153, in filter_traceback..error_handler(*args, **kwargs)
151 except Exception as e:
152 filtered_tb = _process_traceback_frames(e.traceback)
→ 153 raise e.with_traceback(filtered_tb) from None
154 finally:
155 del filtered_tb

File /tmp/autograph_generated_file85bicldm.py:12, in outer_factory..inner_factory..tf__train_step(generated_image)
10 with ag.ld(tf).GradientTape() as tape:
11 a_G = ag__.converted_call(ag__.ld(vgg_model_outputs), (ag__.ld(generated_image),), None, fscope)
—> 12 J_style = ag__.converted_call(ag__.ld(compute_style_cost), (ag__.ld(a_S), ag__.ld(a_G)), None, fscope)
13 J_content = ag__.converted_call(ag__.ld(compute_content_cost), (ag__.ld(a_C), ag__.ld(a_G)), None, fscope)
14 J = ag__.converted_call(ag__.ld(total_cost), (ag__.ld(J_content), ag__.ld(J_style)), dict(alpha=10, beta=40), fscope)

File /tmp/autograph_generated_file1ujc77yi.py:31, in outer_factory..inner_factory..tf__compute_style_cost(style_image_output, generated_image_output, STYLE_LAYERS)
29 J_style_layer = ag.Undefined(‘J_style_layer’)
30 i = ag__.Undefined(‘i’)
—> 31 ag__.for_stmt(ag__.converted_call(ag__.ld(zip), (ag__.converted_call(ag__.ld(range), (ag__.converted_call(ag__.ld(len), (ag__.ld(a_S),), None, fscope),), None, fscope), ag__.ld(STYLE_LAYERS)), None, fscope), None, loop_body, get_state, set_state, (‘J_style’,), {‘iterate_names’: ‘(i, weight)’})
32 try:
33 do_return = True

File /tmp/autograph_generated_file1ujc77yi.py:25, in outer_factory..inner_factory..tf__compute_style_cost..loop_body(itr)
23 nonlocal J_style
24 (i, weight) = itr
—> 25 J_style_layer = ag.converted_call(ag__.ld(compute_layer_style_cost), (ag__.ld(a_S)[ag__.ld(i)], ag__.ld(a_G)[ag__.ld(i)]), None, fscope)
26 J_style = ag__.ld(J_style)
27 J_style += (weight[1] * J_style_layer)

File /tmp/autograph_generated_file1arwgtg9.py:16, in outer_factory..inner_factory..tf__compute_layer_style_cost(a_S, a_G)
14 GS = ag.converted_call(ag__.ld(gram_matrix), (ag__.ld(a_S),), None, fscope)
15 GG = ag__.converted_call(ag__.ld(gram_matrix), (ag__.ld(a_G),), None, fscope)
—> 16 J_style_layer = (ag__.converted_call(ag__.ld(tf).reduce_sum, (ag__.converted_call(ag__.ld(np).square, (ag__.converted_call(ag__.ld(np).subtract, (ag__.ld(GS), ag__.ld(GG)), None, fscope),), None, fscope),), None, fscope) / (4.0 * (ag__.converted_call(ag__.ld(np).square, (ag__.ld(n_C),), None, fscope) * ag__.converted_call(ag__.ld(np).square, ((ag__.ld(n_H) * ag__.ld(n_W)),), None, fscope))))
17 try:
18 do_return = True

NotImplementedError: in user code:

File "<ipython-input-43-b6fcbd5a7ea2>", line 19, in train_step  *
    J_style = compute_style_cost(a_S, a_G)
File "<ipython-input-29-7b56545257a6>", line 30, in compute_style_cost  *
    J_style_layer = compute_layer_style_cost(a_S[i], a_G[i])
File "<ipython-input-24-c133bbc265a0>", line 27, in compute_layer_style_cost  *
    J_style_layer = (tf.reduce_sum(np.square(np.subtract(GS, GG))))/(4.0*(np.square(n_C)*np.square(n_H*n_W)))

NotImplementedError: Cannot convert a symbolic tf.Tensor (MatMul:0) to a numpy array. This error may indicate that you're trying to pass a Tensor to a NumPy call, which is not supported.

I’m not really sure where to begin with it. At first I was having other problems regarding tf and np not working interchangeably. I tried fixing it by using floats but got this error

2

Hi,

Why you are using numpy functions? It is given to you to use the tf, not np?

You may find tf.reduce_sum, tf.square and tf.subtract useful.

Only NumPy arrays are passed to NumPy functions and only TF arrays are passed to Tensorflow functions. We cannot use them interchangeably.

Best,
Saif.

3

And I believe reading these two posts would also benefit you.
Post 1
Post 2

4

The first post seems to show exactly what I did. Which was convert tf.square to np.square. If I don’t use np.square at the compute_style_layer_cost function I don’t pass that function. And if I use np.square there then I have to use np.square later as well. Trust me, if I could just use tf or np alone I would. But compute style layer cost only passes me if I use the np.square with a float value. And then I get the error:

NotImplementedError: in user code:

File "<ipython-input-26-b6fcbd5a7ea2>", line 19, in train_step  *
    J_style = compute_style_cost(a_S, a_G)
File "<ipython-input-14-7b56545257a6>", line 30, in compute_style_cost  *
    J_style_layer = compute_layer_style_cost(a_S[i], a_G[i])
File "<ipython-input-42-d1c305fd2c6b>", line 27, in compute_layer_style_cost  *
    J_style_layer = (tf.reduce_sum(np.square(tf.subtract(GS, GG))))/(4.0*(np.square(n_C)*np.square(n_H*n_W)))

NotImplementedError: Cannot convert a symbolic tf.Tensor (Sub:0) to a numpy array. This error may indicate that you're trying to pass a Tensor to a NumPy call, which is not supported.

under the train step optimizer function.

5

I dont know if you were tagged in my reply. It is below. Thanks

6

You are still using numpy function with TF arrays. Double-check it.

7

When I use tf.square I get this error:

TypeError Traceback (most recent call last)
Input In [118], in <cell line: 3>()
1 ### you cannot edit this cell
----> 3 compute_layer_style_cost_test(compute_layer_style_cost)

File /tf/W4A2/public_tests.py:50, in compute_layer_style_cost_test(target)
48 a_S = tf.random.normal([1, 4, 4, 3], mean=1, stddev=4)
49 a_G = tf.random.normal([1, 4, 4, 3], mean=1, stddev=4)
—> 50 J_style_layer_GG = target(a_G, a_G)
51 J_style_layer_SG = target(a_S, a_G)
54 assert type(J_style_layer_GG) == EagerTensor, “Use the tensorflow functions”

Input In [117], in compute_layer_style_cost(a_S, a_G)
24 GG = gram_matrix(a_G)
26 # Computing the loss (≈1 line)
—> 27 J_style_layer = (tf.reduce_sum(tf.square(tf.subtract(GS, GG))))/(4*(tf.square(n_C)tf.square(n_Hn_W)))
29 ### END CODE HERE
31 return J_style_layer

File /usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py:153, in filter_traceback..error_handler(*args, **kwargs)
151 except Exception as e:
152 filtered_tb = _process_traceback_frames(e.traceback)
→ 153 raise e.with_traceback(filtered_tb) from None
154 finally:
155 del filtered_tb

File /usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/math_ops.py:1516, in _truediv_python3(x, y, name)
1514 y_dtype = y.dtype.base_dtype
1515 if x_dtype != y_dtype:
→ 1516 raise TypeError(f"x and y must have the same dtype, "
1517 f"got {x_dtype!r} != {y_dtype!r}.")
1518 try:
1519 dtype = _TRUEDIV_TABLE[x_dtype]

TypeError: x and y must have the same dtype, got tf.float32 != tf.int32.

8

If while using tf.square and I get that previous error I change the int 4 to float 4.0 I then get this error:

TypeError Traceback (most recent call last)
Input In [120], in <cell line: 3>()
1 ### you cannot edit this cell
----> 3 compute_layer_style_cost_test(compute_layer_style_cost)

File /tf/W4A2/public_tests.py:50, in compute_layer_style_cost_test(target)
48 a_S = tf.random.normal([1, 4, 4, 3], mean=1, stddev=4)
49 a_G = tf.random.normal([1, 4, 4, 3], mean=1, stddev=4)
—> 50 J_style_layer_GG = target(a_G, a_G)
51 J_style_layer_SG = target(a_S, a_G)
54 assert type(J_style_layer_GG) == EagerTensor, “Use the tensorflow functions”

Input In [119], in compute_layer_style_cost(a_S, a_G)
24 GG = gram_matrix(a_G)
26 # Computing the loss (≈1 line)
—> 27 J_style_layer = (tf.reduce_sum(tf.square(tf.subtract(GS, GG))))/(4.0*(tf.square(n_C)tf.square(n_Hn_W)))
29 ### END CODE HERE
31 return J_style_layer

File /usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py:153, in filter_traceback..error_handler(*args, **kwargs)
151 except Exception as e:
152 filtered_tb = _process_traceback_frames(e.traceback)
→ 153 raise e.with_traceback(filtered_tb) from None
154 finally:
155 del filtered_tb

File /usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py:102, in convert_to_eager_tensor(value, ctx, dtype)
100 dtype = dtypes.as_dtype(dtype).as_datatype_enum
101 ctx.ensure_initialized()
→ 102 return ops.EagerTensor(value, ctx.device_name, dtype)

TypeError: Cannot convert 4.0 to EagerTensor of dtype int32

9

You are making the same mistakes again. I think you need to read post 1 again, carefully.

You have to use TF functions with TF arrays and numpy functions with numpy arrays/constants.
Hint: what are the n_C , n_W and n_H? Are they constant or TF arrays?

10

They are tensor dimensions

11

since i’m using tf.reshape they are all tensors. So I should use tf. on them. which I would like to do

12

The key in debugging is to understand what the error message is actually telling you. The problem is that 4.0 is a float and n_C et alia are integers. You want the result to be a float, so how would you remedy that?

13

in post 1 that you shared @paulinpaloalto says not to use tf on the constants like n_C, n_H. to use np.square instead. I’m confused, it feels like I’m being told I have to do two different things. On one hand use np on the constants and not tf arrays. but on the other use tf on all of the arrays in my function. I’m just gonna say I’ve tried BOTH and it hasn’t worked so if tf vs np isn’t the solution then do I need to change the type of n_C, n_H to/from constant/tf array? don’t use tf.reshape? Is the data type of these variables/constants the problem.

14

You have to do something that works. It turns out that the type coercion rules in numpy are more forgiving than they are in TF. It’s perfectly fine to use plain python or numpy for the part of the computation that involves just the scalar constants.

15

Here’s a thread which shows some concrete ways to solve this problem.

16

like doing np.astype(float) to make the result a float?

17

You’re missing the point. 4.0 is already a scalar float. If you’re going to using tf.square for the rest of the computation involving n_C and n_H and so forth, then you need that part not to be an integer, right? That’s what that error message is telling you. But it’s a TF constant, if you wrote it using tf.square. So how do you do that type conversion in TF?

Or just read that other thread I just linked, which shows several ways to do it with np or just plain python for the scalar constant part of the computation. The overall point is that there are a number of different ways to write this that work.

18

when I used tf.cast it didnt work but when i paired that with changing the square terms to **2 it did work…

19

So, the learning is: if simple Python or Numpy can work, then why make life complex.

1 Like
20

I didn’t have any trouble getting it to work by wrapping that tf.square for the scalars with a tf.cast to dtype tf.float32. So that means your implementation must have been incorrect in some way.

But as Saif says, simpler is better. Just using 1. and 2. or 4. to coerce things to “float” and then **2 on the scalar term is way simpler and just works.

1 Like