“With this new dataset LIME then trains an interpretable model, which is weighted by the distance from the sampled instances to the result that we’re interpreting.”
I’m not sure what is meant by “model, which is weighted”. How can a model be weighted ? In what sense ? I understand that neurons/units in a neural networks have weights that are the coefficients in a function that maps inputs to the labels, however I’m not sure what is meant here by a model being weighted.
I’m not sure what is meant by “sampled instances”. Does ‘sampled’ mean permuted/modified as in the feature values that LIME is modifying ?? Does ‘instances’ refer to training data examples or something else ??
Also, I’m not sure whether the “result we’re interpreting” refers to is the new result using the newly-trained model that has been trained with new fake data generated by LIME or to the original result of the original model which we are actually trying to learn about.
Do we need to learn how a new fake dataset makes its predictions and then somehow, that will tell us how our real dataset makes its predictions, which surely could be different from the fake predictions ?
Also, I’m still not 100% clear on
local. Does it refer to ‘all the features’ vs ‘specific individual features’ or to the ‘all examples in a training dataset’ vs ‘subsets/batches of examples’ or something else ?