Fine-tuning the Magenta model for arbitrary style transfer (Fast NST)

Hello!

While tinkering with Lab 2 of week 1, I wanted to fine tune the Magenta model that I load from tf hub but I’m not sure how to go by this. Are there any resources to achieve this? I want to fine-tune the model and increase its accuracy for the images provided in the lab.

Thanks,
Shreyas

1 Like

Hello there,

I am not sure of the magenta model (do you mean the VGG) but you should consider searching and studying transfer learning and I think this specialization has a part dealing with it, either this one or TF Developer specialization. Or you can search the web for transfer learning, I think you will find useful information.

hi Gent

I have the same question here. The Fast NST model loaded from TF hub doesn’t give 100% matching output as described by the paper even though it’s built on the publication. Actually I would say the stylized images are of better quality compared to the ones generated by Magenta model. Would love to find out more about how to tune the model parameters. Is it possible to seek some further advice from TF Hub specialists on this topic?

thanks
Jerry

Hi Jerry,

if you follow the specializations there are sections that describe transfer learning and parameter tuning. In other casers you can find information from where the model is based at (lets say TF HUB) or webpages from people who have done something similar.

The general concept is to use a part of the network with loaded weights and discard the rest, usually the top layers. Then add a top layer (or layers) depending on your output and application and then continue training it. This will improve the existing layers weights (and top layer) to fit the predictions for the application.

This is general procedure, but may differ based on the specifics of the model and where is loaded from, so you need to research for your specific model too.

hi Gent

Thanks for the information. Given the model has already been built and loaded from TF Hub (without options for parameters/hyperparameters tuning), I will try to follow the paper and rebuild the model from scratch. The only downside is the content/style weights weren’t mentioned in the paper, but will give it a go!

Best
Jerry

1 Like