Week 1 Assignment 2 - Functional API - Convolutional Model

[bishopb]

Hi there,

I am unable to find the bug in my code for the “convolution_model” function assignment.

For my second ReLU layer is outputting the shape (1, None, 8, 8, 16) instead of (None, 8, 8, 16), which is causing an error in the tests as it is the wrong shape.

I have read the docs and tried adjusting various parameters of the ReLU layer and the layer before but I have been unable to solve it. The definition of this layer is A2 = tf.keras.layers.ReLU()(Z2) .

It is confusing as according to the docs the output shape of a ReLU layer is supposed to match it’s input shape, and the input to the ReLU (i.e. the output of the previous layer) is the correct shape of (None, 8, 8, 16).

A potential clue as to what the problem might be is that although it has the correct shape on the output (screenshot below)the object returned from the previous conv2D layer (object ‘Z2’) is for some reason a tuple and so has no “.shape” property, whereas in all the documentation for conv2 layers (and the previous conv2D layer in my own code) the output should have a shape property, i.e. should not be a tuple. So perhaps the problem lies with that prior layer, as I cannot see a problem with the ReLU layer. I cannot understand why the Conv2D output is in that format and I can’t find anything in the docs that would explain it so feel I am missing sometime. This layer is specified by Z2 = tf.keras.layers.Conv2D(filters = 16, kernel_size = (2, 2), strides = 1, padding = "SAME")(P1),

All other layer shapes are correct and the tests pass for the previous layers (including the conv2d layer in question)

Many thanks in advance for any help.

edit - code snippets removed

ps i have attached the 2 errors I get below:

[TMosh]

Posting your code on the Forums violates the course Honor Code.

[bishopb]

I have removed the large snippets. Very sorry I was trying to make the problem clearer for anyone trying to help. If the pictures of the errors are not ok I can remove them too. I hope the problem is still clear. Likewise if the small snippets I have left in to show where I think the problem is then I can also remove these. As these parts are not working I hope it does not count as sharing a solution.

[paulinpaloalto]

You’re going to scream when you hear this. I think the problem is that you added an extra trailing comma at the end of the line on the previous Conv2D layer. (I can see it in the exception trace that you gave.) That syntax in python turns the RHS of that assignment statement into a tuple. When I artificially introduce that bug in my code, I don’t get exactly the same error that you get, but it does fail spectacularly. My guess is that is at least part of the problem. Maybe you’ve got some other syntactic irregularities like that which also contribute.

Well, looking more closely at my error output, I do get the error about the shape from the ReLU being [1, None, 8, 8, 16], but I get it on the max pooling layer. It looks like maybe you somehow truncated everything at that point.

So I think that actually is the problem. Programming is a game of details. The difference between ; and : can spoil your whole afternoon. Or a stray , where you didn’t mean to put one …

[bishopb]

Paulinpaloalto thank you so, so ,so much for taking the time to look at this and to find the problem which was exactly as you identified. I had two random trailing commas after different layers in my code and they were the source of the problem - it is all working perfectly now.

I had no idea about that tuple-ify trailing comma python syntax so I am very glad to learn it, although it is painful that this was the problem after I spent so long diving into the keras docs! I was so focussed on the shiney new methods I was using I forgot to consider everything else. A very good lesson learned and thanks so much helping me learn it. I don’t think I would have noticed that in a whole week of looking so you have saved me many extremely frustrating days.

Thanks once again.

[paulinpaloalto]

Hi, Raymond. I’ve seen that kind of bug before, but I admit that it took me a while to find it in this case. It’s especially hard to see because there are some TF syntax instances in which you need the commas (e.g. declaring a Sequential sequence). There were a few other differences between the way you wrote that line and the way I did, but trying your variants made no difference. Fortunately the exception trace showed that full line of code and finally I managed to notice the comma. I’m glad that was the answer!

Cheers,
Paul

[bishopb]

Yes I think that the commas appeared because I (very lazily) copied the code from the Sequential sequence in the sequential API task and then adapted it, and just did not even consider the commas as being able to affect the output that way. It is a really good thing to know about in python though because it is definitely something I could have easily done again and not knowing about that operator I would always have been looking problems elsewhereso thanks once again - it is a great thing about this forum that you can get practical help like that.