I have some doubts regarding the deployment of a recommendation model. My main concern is whether I need to retrain it every time there’s an update in user data, since to provide new recommendations, it seems necessary to update the model.
How could I train a model and generate new recommendations for the same user without necessarily having to retrain it?
A real-time method would allow for the model to be updated on every change in the user data.
Hey TMosh, thank you for the answer !. However, my question is how big tech companies like Amazon, Netflix, etc., sustain their real-time recommendation models because it’s not feasible to retrain the model with every user interaction with the product (for example, running the algorithm with every user click, or retraining the model after every movie watched on Netflix).
I am having trouble figuring out how to put my recommendation model into production using MLFlow in a way that supports incremental training or enables different recommendations for each new user interaction.
How could I have a trained model and use it multiple times using the MLFlow API, similar to how I would for models from XGBoost, decision trees, etc.
As I said there are several real-time methods that update after every new data observation arrives.
For example, stochastic gradient descent.
Sorry, I’m not familiar with MLFlow.
This must be a topic on which papers have been published. I just googled “incremental training or online learning for recommender systems” and got this as the top level hit. I’m sure there are more articles and probably a YouTube search might also turn up some lectures on this.
I have not taken MLS and don’t know if they cover recommender systems there, but Prof Ng did cover them in his original Stanford Machine Learning course and I think I remember he did address the question of how to update models. But I think he just talked about it in general without going into too much detail. But it’s been 6 or 7 years since I took that course, so my memory may not be reliable on that.
@TMosh @paulinpaloalto Thank you for the help!