In the context of data preprocessing for machine learning models, how do normalization and scaling techniques differ in their approach and impact on the dataset? Specifically, when should one prefer normalization over scaling, and vice versa, considering the nature of the data and the specific requirements of the machine learning model being used?
Thank you all,
Normalization is a one of the ways of feature scaling. There are other forms of feature scaling, such as standardization.
Like you said, which one you should use depends on multiple factors, such as which AI model you’re using, the distribution of the input data, etc. When developing an AI model, it may be worth trying different types of feature scaling to see if it makes a difference.
It’s a longer subject, and if you want to learn more, there should be plenty of resources online about it. I think this one is pretty good:
Let us know if you have a specific problem you’re facing, and we might be able to provide advice for that specific case.
Thank you but ‘Normalization’ is asociated with Regularization, but Scaling is not as long as i have understood on C2W1 of ‘Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and…’
I am not understanding this differences
Normalization and Regularization are different techniques. The two are unrelated, even though they are explained in the same C2W1 week. You can choose to apply either one, both, or none to your AI model.
Normalization is one way of Feature Scaling. There are other ways, such as Standardization. It involves modifying the inputs into an AI model so the values are within some ideal range.
Regularization is a different technique. The purpose is to prevent the model from overfitting when training. There are many ways to do this as well (L2 Regularization, Dropouts, etc).
Aside: One terminology trap to watch out for is you might hear “L2 norm” when people talk about regularization. This is not the same as normalization in feature scaling. It’s an unfortunate re-use of the same word to mean different things.
Thank you!!
The difference is, in scaling, you’re changing the range of the data, while in normalization, you’re changing the shape of the distribution of the data.
In both cases, the numeric variables are transformed so that the transformed data points have specific helpful properties.
Feature scaling and normalization are important steps in image recognition, as they can help to improve the performance of machine learning models and as they help to ensure that all of the features in an image are on a comparable scale.
Regards
DP
Thank you!
Scaling: change of Range of the data
Normalization: Change shape of the distribution of the data
Normalization and scaling is not correlated to Regularization, but how feature scaling and normalization affect regularization is more important to understand.
As explained by @hackyon regularization is a technique to prevent overfitting in neural networks and thus improve the accuracy of a Deep Learning model, so one needs to understand normalization and scaling need to done before regularization in a way that it does not penalises any larger parameters. That is the only relation between these.
Scaling is a less complete type of normalization.
There’s little reason to not always use normalization.