DLS, CNN, Week 2, Assignment 1 - Parameters order matter in Add command

Course Q&A Deep Learning Specialization Convolutional Neural Networks
1

Little comment about the first assignment of week 2 on convolutional networks.
I spent quite some time understanding what was wrong in the architecture of the ResNet, and in a way that gave me a good opportunity to study the feature and parameters size of each layer, so I see it as a good thing.
However, some people may struggle…

When building the identity and convolutional blocks, the following comment below implies to add X_shortcut and X with X_shortcut being the first element in the Add function.
However, when doing so, this will cause comparator(summary(model), ResNet50_summary) to generate an error.
The reason is that the Add command should be called with X being the first parameter, and X_shortcut being the second parameter.

So: X = Add()([X, X_shortcut])
And not: X = Add()([X_shortcut, X])

2

Yes, that is a good point. Of course we know that Addition is Commutative, so the actual results don’t change depending on the order of the operands. But the test for resnet50 uses the comparator function to literally do a text comparison between the “summary” output for your model versus their correct solution. If you change the order of the operands, then the compute graph gets printed in a different order and the line by line comparison fails, even though the two models are functionally equivalent.

But notice that the addition code was just given to you in the template code for convolutional block. And changing the order in the identity block does not cause the test to fail. So if it failed for you, that means you went out of your way to create the problem by changing the template code that had been given to you. In the original version of this assignment, the addition was code you needed to write in both cases, but they changed it to make that part of the template code precisely to avoid people running into this problem.

3

Thanks for your quick reply.
I don’t have to go out of my way to create a problem :wink:

The test functions public_tests.convolutional_block_test and public_tests.identity_block_test will pas whether I write
X = Add()([X, X_shortcut]) or X = Add()([X_shortcut, X])

Now, if I call comparator(summary(model), ResNet50_summary) with the wrong parameter order in the Add command, I will get the following error:

Test failed
Expected value 
 ['Conv2D', (None, 8, 8, 512), 66048, 'valid', 'linear', 'GlorotUniform'] 
does not match the input value: 
 ['Conv2D', (None, 8, 8, 512), 131584, 'valid', 'linear', 'GlorotUniform']

Simply reversing the order of the parameters in the Add command solve the issue, and the test passes.

4

Sorry, but when I reverse the order only in the identity_block, the tests all pass.

It is only when I reverse the order in the convolutional_block function that the tests fail.

I just carefully reran all the above tests before I sent my previous post.

And the point I made above is that the code for the addition in the convolutional_block was just given to you. So the only way to create the problem is to rewrite correct code that was given to you.

1 Like
5

I must have copy-pasted the Add and Activation lines of code from the identity block to the convolution block.
Thanks for your patience!

1 Like