In most ML materials, we usually see the categories: supervised, unsupervised, semi-supervised, self-supervised, and reinforcement learning.
So where exactly does a recommender system fall under this framework?
Should we treat “recommender systems” as another separate category on its own (so it becomes supervised / unsupervised / semi-supervised / self-supervised / reinforcement learning / recommender systems)?
If we classify purely by learning type, is collaborative filtering supervised, unsupervised, or something else?
I’m also confused about how the boundaries between these learning types are defined.
For example, in the anomaly detection lesson:
The training set only has features and no labels → looks like unsupervised learning.
But later in the videos, the validation and test sets do have labels.
So in this case, should the model be considered unsupervised or semi-supervised? Since part of the data (training) has no labels, but another part (validation/testing) does.
Does it actually matter which category of machine learning something belongs to?
In my opinion, if the training process involves labels, that is supervised. In other words, for the anomaly detection algorithm in the lecture, those trainable parameters in the Gaussian were trained unsupervised-wise. As long as those parameters are not changed in the validation/testing phases, I think those phases don’t matter as to what kind of learning those parameters are under.
You might tune the epsilon value (threshold below which a sample is considered abnormal) in the validation stage using the labels, but is that tuning process machine learning at all? Looks like a human tuning to me. If it’s not machine learning, do we need to say it is supervised or not?
Following my idea above, as for whether a recommender system would be considered by me supervised or unsupervised, it depends on the actual training process. If you frame the collaborative filtering problem like the assignment does which is a neural network that takes the inputs user ID and item ID and produces output which is rating, then training such network with the ratings looks like a supervised learning to me. If you frame it as a matrix decomposition problem then I will call it unsupervised.
I would not say that because the ratings are considered labels in the neural network setting, they will be considered labels in any other settings.
When I come across a machine learning problem, I won’t think about which category it belongs to, so I think it’s not important. However, mastering different types of techniques is important to flexibly apply the skills in a new problem. So, I won’t recite which work is supervised/unsupervised/…, only to think about if I can do the work supervised-wise/unsupervised-wise/…
Anomaly detection does use a labeled training set - it’s just very small compared to other supervised methods.
It starts with a large set of unlabeled data, so the statistics can be determined.
Then it uses a small labeled set in order to adjust some anomaly detection threshold based on those statistics.
So I’d agree it isn’t just a pure unsupervised method.
== = = =
I agree that it really doesn’t matter how you group the machine learning methods. You just need to know what methods exist, and what situations they are intended to model - and what situations a particular model is not well-suited for.
For example, you’re not going to use a deep fully-connected neural network for a situation where the sequential order of the examples is important.
