For example, I will take some pretrained model like llama 2 70B and will train it for my task. After some time there will be llama 3 70B and I will want to update my current model. But it has a lot of corrections during my own tasks. So, what should my actions be? Should I fine tune my old model, or should I fine tune llama 3 model?
Also, what if I will change my mind about uning large models and will need to use any 7B model to save money on servers? Are 70B models and 7B compatible? What and to what should I fine-tune in this case?
Hi @someone555777
These kind of questions are answered in MLOps Specialization.
Also, there are many blog posts online about this, like:
Cheers
I’ve ended mlops specialization and I don’t remember that we worked with fine tuning in details 
I remember just a lab, where we learned the deploying of a few verison of models in the same time (Model Versioning).
Maybe this is part of Knowledge Distillation topic, but in general, I think, it’s not my case. It is the case just partly maybe if I would like to transfer knowledge from 70B model to 7B.
And we learned about Model Decay. It is partly about this topic too. But it’s just that we should relearn a model and maybe to change its architecture in outcases.
Maybe Transfer Learning in Deep Learning Specialisation, but we just learned, how to define an external model as base for the model, that I create for my purpose. But we didn’t learned, how to update the state of this base model, when it was updated its version, for example.
But in general, I don’t remember, that we learned the working with different types and versions of external ready models at all in all courses, that I passed.