Neural Style Transfer code error

Course Q&A Deep Learning Specialization Convolutional Neural Networks
1

help!

{moderator edit - solution code removed}

error is: np.isclose(J_style_layer_SG, 14.01649), “Wrong value.”

2

They explained in the instructions why it does not work to just reshape directly to the final shape you want. It does not preserve the “channels” dimension. Here’s a thread which explains this in more detail. Also check the earlier post on that thread.

BTW this is not the Transfer Learning assignment, but the Neural Style Transfer one, meaning that the title of the thread is misleading.

1 Like
3

thanks I got it,
by the way, i want to ask when we are doing style transfer, in each epoch the gradients only update the generated image, does it mean the deep learning architecture parameters (such as W and b in VGG layers) keeps the same along the whole training procedure?

4

That’s an interesting question. Yes, I’m pretty sure that the intent of all this is just to use the VGG network “as is” to extract information about the three images we are dealing with (content, style and generated output). I think it’s correct to say that the gradients are being applied only to the generated_image tensor. Notice that they don’t do the usual style where they define a model and then just do “model.fit()”. They manually use tf.GradientTape to compute the gradients and then apply them to generated_image. Of course without really digging into the implementation code, it’s a little hard to be completely sure what’s happening inside that apply_gradients call. Gradients are probably being computed for everything in the compute graph between the input and the cost, which does include the VGG parameters for the selected layers that we are using to compute the cost. But then the question is whether they actually get applied to the VGG layer parameters. One way I can think of to confirm or deny this theory without the exercise of trying to understand all the relevant TF source code would be to sample the VGG layer parameters for one of the layers we are using before and after one call to train_step and see if they change or not. Not sure I’ll have the time to do that today, but I promise to at least think about doing it. If you get ambitious to try it and get there first, please let us know what you find.

Or if someone else who has a better understanding of the TF functions sees this and knows the answer, please chime in!

5

Ok, sorry it took me a while to get around to this, but I did finally write the code to look at the input saved model and compare the weights for one of the layers we actually use (block5_conv1) after the training. As we expected, those weights did not change. Here’s the code that they use to load the VGG model:

tf.random.set_seed(272) # DO NOT CHANGE THIS VALUE
pp = pprint.PrettyPrinter(indent=4)
img_size = 400
vgg = tf.keras.applications.VGG19(include_top=False,
                                  input_shape=(img_size, img_size, 3),
                                  weights='pretrained-model/vgg19_weights_tf_dim_ordering_tf_kernels_notop.h5')

vgg.trainable = False
pp.pprint(vgg)

You can see that they set “trainable” to False, so it all makes sense. But I actually did compare the weights before and after training to confirm that it works the way we would expect.

1 Like