I am a bit confused about how the get_score function will work in the big picture of the training process.
The idea is for us to apply a penalty to the features that are changing, and then use it to diminishes the value of the target_score feature. The higher this difference is, the better.
I would like to understand better, how this score calculation is helping the model to:
- classifying correctly the non_target features
- changing the noise vector to produce the target_feature well enough to actually be classified corrected in the end.
Thanks in advance for any clarification.
Hi Jaspier!
Hope you are doing well. I will try to give some intuition to you. But it will be a long story, sorry for that.
-
The get_score function is designed to calculate a score that evaluates how well the model is performing with respect to classifying the target feature while penalizing changes in the non-target features. This score is then used to guide the training process.
-
The purpose of penalizing changes in the non-target features is to encourage the model to focus on isolating the target feature and minimising the entanglement between different features. By penalizing differences from the original class using L2 regularization, the function aims to minimize the changes in the non-target features when modifying the noise vector.
-
The calculation of the other_class_penalty takes the differences between the current noise and the old noise for the non-target classes. The norm (magnitude) of these differences per example is then computed, and the mean of these norms is multiplied by the penalty weight. This value is negated since it is a penalty. The resulting other_class_penalty represents the penalty for changes in the non-target features.
-
On the other hand, the target_score is calculated by taking the mean of the current classifications for the target feature. This represents how well the model is classifying the target feature in the current noise.
-
Finally, the target_score is added to the negated other_class_penalty to obtain the overall score. The higher the score, the better the model is performing in terms of classifying the target feature while minimizing changes in the non-target features.
Why higher the score, the better the model?
We are adding target_score to the negative of the other_class_penalty. ( T - P ). We want target_score to be more and Penalty to be less → T - P should be higher ( T is a high positive number, P is a small positive number thus T - P will be higher in that manner).
Thus we are making sure that other features don’t change much while the Target features change.
Hope you get the point, if not, feel free to post your queries.
Regards,
Nithin