In the supervised ML course (I am just on the first week) Uni-variate Linear Regression, Multivariate Linear Regression, and Logistic Regression are covered.
In the case of Uni-variate and Multivariate Linear regressions, ( not sure about Logistic Regression) it is not necessary to use Machine Learning for these techniques. For example, it is possible to compute an estimate for Uni-variate Linear regression using the OLS estimator.
So when is it necessary to use Machine Learning to fit the model (instead of using closed form formulas with the help of a computer) ?
Also, when is it better to use Machine Learning, and what are the benefits?
Closed-form formulas work well for simple tasks, but as complexity increases, machine learning algorithms step in, handling intricate relationships and larger datasets effectively. When finding answers becomes challenging with closed-form solutions, iterative algorithms in machine learning provide a powerful alternative.
Hope this helps! Feel free to ask if you need further clarification
@Sam_R I would also say the complexity of the problem and the amount of data you have also figure into the consideration of what model to use. For example if the function on your model is obviously not linear/or in the case of say data separation/classification not linearly separable that OLS won’t work well.
However, in later courses such as when you learn about Neural Nets, you’ll find that most such models require a lot of data to be trained and effective, and if such data is not available, other ML methods (ex Support Vector Machines (SVM), K-Nearest Neighbor (KNN), etc, and even various forms of regression) may work better. They may also be more effective when your compute time/limitations are restricted (or when something like a Deep Neural Net would just be ‘over kill’ for the problem you are trying to solve).
I think one of the important thing to learn is that in the end, while certain models might be generally better at solving certain types/classes, in the end there really is no ‘magic’ model that does amazingly well on all types of problems, thus they are all worthy of knowing/having in your tool kit.
The closed form solution (analytical approach) to the unregularized linear regression problem requires X^TX to be invertible, so if it is not, you might have to use instead an optimization approach, such as gradient descent which is taught in this MLS. If the dataset is too large that computer’s memory can’t handle these matrix algebra, optimization becomes necessary too.
