Hey!
I just finished the assignment, but I am not sure that I understand completely how we calculate the get score function. I dont understand why we calculate the penalty for each sample, instead of calculating it for each feature?
That is the way I calculate my function:
other_distances = original_classifications[:, other_indices] - current_classifications[:, other_indices] #print(other_distances)
# Calculate the norm (magnitude) of changes per example and multiply by penalty weight
other_class_penalty = -1 * torch.norm(other_distances, dim=1).mean() * penalty_weight #print(other_class_penalty)
# Take the mean of the current classifications for the target feature
target_score = current_classifications[:,target_indices].mean()
But here, the "other_class_penalty " value is going to have one value for each sample, but for me it would make more sense if it has one value for each feature. Correct me if I am wrong, but the current way only incorporates that the mean of the feature to be correct. On the other hand, if two features always change the same way (lets say one has a high, and other has a low value), than it would not be visible in the mean, therefore it would not be penalized?
Or am I missing something here?
Also, I have an additional question. I have read the explanations about how we learn the features, yet they are still not clear for me. If someone would explain to me, how to we learn the features by updating the nosie? I see, that we do backpropagation on the “fake_classes_score” but how does it help us? Laso how is the noise going to have a gradient?
Wouldnt it be smart to fine-tune the generator rather? So that it learns also which features it should pay give higher attention from the noise? I am very lost here.
Hi @Peter_Szabo,
Great questions! I’ll try to answer as best I can.
For your first question: I think calculating the penalty for each sample instead of each feature was just a choice of one intuitive way to look at it, with the reasoning going something like this: For each image we want to calculate a score that considers how much we’ve changed the feature we want to change (like “smile”), but we want to add in a penalty if we’ve also changed other features. Since we are using norm (magnitude), the penalty for each of these other features will always be positive (or 0), so we never have the situation where a negative value for one feature might cancel out a positive value for another feature. If one of these other features has a large change and another has a smaller change, then we’ll have a penalty that takes into consideration that we have one large plus one small change.
With this reasoning, we calculate a penalty for each sample, and then average across all samples. But that’s just one way to think about it. You could also look at it feature-by-feature as you suggest. In fact, I think you’d come up with the same result if you took the norm of each feature across all images, and then took the mean of all of those results. It seems like the same math. Either way, you’re calculating a norm value for each image-feature pair, and then taking the mean of all of those values. You can give it a try by changing the code to do it the other way - maybe do both calculations and then just print out the result of the ‘other_class_penalty’ using each technique to see that they are the same.
For your other question about what’s generally going on: The main steps are that we feed random noise into the generator, the generator generates an image from this, and then the classifier determines the features in that image.
In this assignment, the generator and classifier have already been trained, so what we are doing is starting with some random noise, then using the generator to create images and classifier to tell us what features those images have. Then, we try to tweak the noise a bit to get our generator to create slightly different images with more of our target feature.
I hope this helps, but if I’ve missed the point or overlooked something, let me know and I can try to fill in the gaps.