Week 4 Assignment 2 Exercise 6 Issue

Course Q&A Deep Learning Specialization Convolutional Neural Networks
1

I’m currently working on Week 4’s Assignment #2. I am having an issue with exercise #6, the train_step function. I am receiving an error stating that the value of J is a really negative number. I suspect I may have an issue with another function, but I am too sure as all other functions were checked successfully. Here is the error I am receiving:

tf.Tensor(-128130743.6410612, shape=(), dtype=float64)
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-186-e8adb96bb73f> in <module>
      5 print(J1)
      6 assert type(J1) == EagerTensor, f"Wrong type {type(J1)} != {EagerTensor}"
----> 7 assert np.isclose(J1, 10221.168), f"Unexpected cost for epoch 0: {J1} != {10221.168}"
      8 
      9 J2 = train_step(generated_image)

AssertionError: Unexpected cost for epoch 0: -128130743.6410612 != 10221.168

Here is my code for exercise 6:

{moderator edit - solution code removed}

Any help would be much appreciated!

2 Likes
2

So, it looks like I had to convert my values from ‘UNQ_C3’ and ‘UNQ_C1’ to python float values as opposed to TensorFlow objects to fix this issue.

1 Like
3

I got the same issue. It seems that it is, as you said, related to type conversions.
In my case it was in one of the previously implemented functions which is UNQ_C3 (compute_layer_style_cost function). There I used tf.square to square my numbers which produces a tensor variable and not a plain data type. This affects the result type of the equation that it is part of. I replaced the usage of tf.square with the plane x**2 operation and my error was fixed.

This is what I mean:

if n_C is 3 then

n_C ** 2 will be: 9

tf.square(n_C) will be: tf.Tensor(9, shape=(), dtype=int32)

6 Likes
4

Hi. so I have the same issue: I did what you suggested, I think, but still not fixed:
This is my UNQ_C1 and UNQ_C2. They both pass their own tests, but at UNQ_C5 I get the same error as OP mentioned.

{moderator edit - solution code removed}

1 Like
5

try using tf.square and tf.subtract (as suggested in the “additional hints”) when dealing with tensors.

1 Like
6

Hi, I had this issue and the discussion here was very helpful for fixing it. Do any of you feel you understand what is happening here?

I’m thinking it’s behaving as if using tf.square(n_C) causes a trainable variable to be created, after which everything goes haywire. But that would be a weird thing for TF to do, and I’ve not been able to reproduce the problem in a simple standalone example.

1 Like
7

I have the same issue and I still didn’t realize where is my error !!
I used all tensor flow functions for calculations ex. tf.square, …
any suggestions ?

1 Like
8

If you have the same problem I had (in the definition of compute_layer_style_cost),
then it goes very wrong if you write something like:

tf.square(n_H * n_W * n_C)

but seems to work if you write

(n_H * n_W * n_C)**2

I have no idea why, but would love to know.

2 Likes
9

Just to add to that last message, the Coursera lab for this exercise runs Tensorflow version 2.3.0.

I’ve now rerun this on a local PC with Tensorflow 2.7.0, and the code which didn’t seem to work for me in the lab…

... tf.square(n_H * n_W * n_C)) ...

… seems to work perfectly well in this environment.

I’m wondering if it was some Tensorflow issue that has now been fixed.

1 Like
10

Hello!

I faced the same error. There were few different comments about type of values that should be there, I used them, but it didn’t help me. So, if someone will be struggling with this, I’d like to recommend do the same as me:

  1. Use only suggested in exercise tf.functions, for the rest use python operators (+, -, *, etc.).
  2. There are specified dtype for output values in UNQ_C5 - I think it could be treated as some advice, i.e. your graded functions that is used in UNQ_C5 should return tf.float32 dtype for output values

Taking into account these points helped me to pass tests, hope it will help someone else :slight_smile:

1 Like
11

What I noticed was that you have to re-run all cells from UNQ_C1 after each time you run UNQ-C5, then the error does not occur. And I’m using all tf functions including the function ‘tf.math.squared_difference’ which is handy but not one of the recommended functions. I passed all local tests and 100% on the submission.

1 Like
12

I think this is the correct issue. The seed() command appears earlier and re-evaluating some cells makes the changes the internal state of the random number generator.

1 Like
13

Indeed as Valentino said
restart the kernel and it will work fine

1 Like
14

The real problem with getting negative cost values is that mixing integer and float types in TF/Keras has different type coercion rules than it does in python. Here’s a thread which gives details and examples.

1 Like
15

I got the negative result from C4W4A2 exercise6. I take care of the type conversion and use the recommended function tf.substract, tf.square and tf_reduce_sum. I also reset the kernel before running EX6. but I am still getting the negative result. Here is the pointer to my area

I am stuck and appreciated if someone can help. I see these error.

c4w4a2_ex6
c4w4a2_ex61030×426 27 KB

1 Like
16

I solved this problem by rewrite the following line
From
J_style_layer = tf.reduce_sum(tf.square(tf.subtract(GG, GS))) * (1.0 / tf.cast((4 * tf.square(n_H*n_W*n_C)), dtype = 'float32'))

To
J_style_layer = tf.cast((1.0/(4*(n_H*n_W*n_C)**2)), tf.float32) * tf.reduce_sum(tf.square(tf.subtract(GS,GG)))

3 Likes
17

I’m glad to hear that you found a solution. Not sure if you found this thread in your searches, but it’s my best attempt to document what works and what doesn’t.

1 Like
18

Thank you for the reply. I read your document. In my case, I thought that tf.square should be used and it caused me problem. Once I replaced tf.square with **, it worked well.

2 Likes
19

The problems appears to stem from operations on integers. For me, the definition of scaler in exercise 3 was the issue.

This definition did NOT work

scaler = tf.divide(0.25,tf.cast(tf.size(a_S) * tf.size(a_G),dtype=tf.float32))

But casting each integer value did work

scaler = tf.divide(0.25,tf.cast(tf.size(a_S),dtype=tf.float32) * tf.cast(tf.size(a_G),dtype=tf.float32))

20

Interesting. It’s good to hear that you got to a solution. My only concern is that I honestly don’t see how the change you show could account for a difference in behavior. Both the inputs are integers. Why does it matter whether you do the “cast” before or after the multiply? My theory is that something else changed at the same time that actually accounts for the difference.

Did you read my example thread (linked several times earlier on this thread) where I showed some of the possible ways to get errors here?