Neural Style Transfer - multiple style images

I tried to take the last exercise in C4W4 one step further and try to compute the style based on multiple images. What I thought was to get the average per set of activation layers ( avg(a_L_img1, a_L_img2 …) ) in order to keep the a_S with shape (n_C x n_C). It seems that I don’t get great results just yet :slight_smile:

I was wondering if anyone tried something like this and if she had different approach maybe.

Thanks.

Hi, Gabriel.

It is cool that you are trying experiments to extend the ideas given in this section. That’s usually a great way to learn new things and extend your intuitions.

I don’t have any experience trying anything like what you suggest, but maybe you could say a little more about what you are trying here. Do you mean having multiple “style images” and averaging the cost across all the style samples? That sounds like it should work in principle, but then it probably matters quite a bit how you pick the style images. E.g. do you use multiple images from the same painter (Monet in our case), so that there is some commonality in the styles? Or do you pick one from Monet, one from Picasso, one from Van Gogh and one from Rembrandt? If the latter, then it seems reasonable to expect that you might get pretty mushy results. What is the average of Rembrandt and Van Gogh? :scream_cat: :laughing:

Hi Paul,

Yes, ofcourse I didn’t mix Monet and Picasso :smile:
I took multiple Monet paintings (although at the beginning I had also some paitings mis-labeled as Monet) and I thought that in theory it should develop a “better” style matrix (by averaging the related outputs). I tried to choose similar paintings but it’s true they are not perfectly similar. I noticed that using (only) the Monet proposed in the course, it took the poppy flowers as a style feature rather than objects in the paiting. The result was having red poppy like features in my Louvre result image. I thought that with multiple paintings, the poppy flowers will have a reduce significance in the overall style.
Nevertheless the results with my multiple painting style, were not great.

Another thing I tried was to vary the choice of style_layers. I have the feeling that getting one layer from each block it’s maybe not the optimal choice. I tried to get rather more low level layers, hoping to catch more basic patterns from the Monet paintings. At the same time, using Picasso paintings, which are more abstract, maybe another choice of layers would be better.

Thanks for the next level of detail on what you tried. That’s a great point about how it seems to home in on the red dot pattern of the poppies as the thing it considers “style” in the single image case, although it does sort of blur things a bit in a general Impressionistic way. The other worked examples they show us in the notebook with Van Gogh and others, it seems to take in more of the features of the style images. I wonder if they used different layers in those cases.

Other hyperparameters that might be worth some experiments would be number of iterations and see if the convergence behavior has much impact. Thank you for sharing your results! Keep us posted on what you learn. I hope your experiments will inspire others to try things like this.