Hi all,
I really like the idea of Neural Style Transfer and would like to create my own. In order to do this, is there a way I can download the code to the VGG19, or can I just run: tensorflow.keras.applications import VGG19, then set include top to false and trainable to false? After doing this I am assuming I can follow the steps used in Course 4 Week 4 of the deep learning specialization to make this my own model? Also, when running this in VS Code, would a laptop be able to run a VGG network with more than 19 layers? This is a lot of questions, and I would appreciate any help at all!
1 Like
Hi there,
regarding the NN w/ 19+ layers, I believe this can be borderline but it will depend on your specs of your hardware and the data itself if it will work in a reasonable time on your laptop or computer.
Feel free to consider the use of GPUs. If you do not want to buy new hardware of course you can also ârentâ it in the cloud, e.g. via:
- Google Colab
- or cloud providers like Azure, AWS, Google Cloud
Side note: content-wise I did something similar concerning style transfer learning, see this repo w/ a MacBook Pro (Core i5 2320, 8 GB RAM) and in order to create one picture which you can see in the repo, training took about 10h + which is quite a lot.
Both pictures: style and content were taken with an iPhone camera and were not edited before training.
Hope that helps! Good luck with your project, @GageMars!
Best regards
Christian
Itâs definitely worth trying to run the Neural Style Transfer on your local computer. What happens in the notebook is pretty complicated, but if you look closely youâll see that we donât actually train VGG-19 in order to accomplish the style transfer. The training just works on the actual images themselves and is just using VGG-19 in inference (âpredictionâ) mode. I havenât looked at the details of the layer sizes in VGG-19, but normally itâs training thatâs the expensive thing because of the huge dataset sizes that are required. Just running the trained model shouldnât be a problem.
Side note: I used gradient descent w/ Adam for the optimization problem w/ an overall loss which combined the weighted style loss and the content loss as suggested in the original 2015 paper, see formula 7, starting from a white noise picture on my machine (see HW specs in post above).
Also here you can find a nice step-by-step explanation by TF.
I would be quite interested how your training performance on your local machine is and how many pictures you want to âstyleâ, @GageMars.
Feel free to keep us posted!
Best regards
Christian
Correct. Like in the notebook in the assignment, we imported the VGG model from keras then âfrozeâ the layers to use their pretrained weights. We then set include top to false, and we created our own loss and last layers. Is this the process I would use to create my own without training an entire model?
What you are describing would be the process to do Transfer Learning with VGG-19, but that is not really what is happening here. Take another look through what happens in the notebook. Notice that we are not adding any new output layers to VGG-19. What we do is use the existing trained VGG-19 model in âinferenceâ mode and then feed it three different images: the âstyleâ image, the âcontentâ image and our new âgeneratedâ image. Then we extract the activation outputs produced by a selection of internal hidden layers of VGG-19 on those three images and we define two different cost functions based on those hidden layer activation values. The overall loss is defined as a weighted sum of the two style and content losses.
Now look at how the training works: we are not training anything about VGG-19 or any additional layers added to that. We are applying the gradients generated by our custom loss function directly to the generated image.
At least thatâs my reading of the code âŚ