I want to visualize the cost function of the kmeans problem, so that I can see how it has local minimas and how the algorithm might get stuck in it, but I’m still not sure how to do that.
I tried generating simple uniform 1-d data and calculate the cost for every 2 pairs of possible centers, but the resulting cost function looked convex to me (even though it had like 2 global minimums but that because the center got flipped I suppose since it is a symmetric function).
I guess if the data is uniform then the algorithm always converges? (makes sense in a way), maybe if I try it with only 2 centers but the data has like 3 centers in reality? like it is dense in 3 regions instead of only 2 maybe? or are there other scenarios that I can try?
Also, I suppose it is impossible to visualize the cost function for 2-d data? or there are tricks which can be done?
K-means doesn’t really have a ‘cost function’ because it isn’t a supervised method.
We’re just assigning examples to the nearest centroid using the least squares distances.
There are no local minima.
Just because it is not a supervised learning problem doesn’t mean that there is not cost function no? It uses " WCSS (within cluster sum of squares)", which is the objective function we are trying to minimize (I thought that objective and cost functions are the same but I maybe wrong). This is the functino that I would like to visualize.
And professor Ng said in the lecture “Initializing K-means” does say that the algorithm we use to optimize the objective function might get stuck in a local minimum. He did give some example with a 2-d dataset and k = 3 in around the third minute of the lecture, and it is easy to replicate such a scenario but I don’t know how I can visualize the function for a problem with such dimensions.
You cannot effectively visualize any function that has more than 3 dimensions.
That’s why I tried to visualize it for 2 centers only; with 1-d data, but I got a convex function for the sample data that I used
Convex functions are good, because they have no local minima.