Thanks for adding this.
There are often questions in these fora about whether one can understand AI without knowing the math behind it. To me, this demonstrates that the answer is ‘No’. I think to truly understand you have to not only know mechanics like matrix algebra or what is a derivative, you have to be deeply fluent in the concepts; maybe you have to be able to dream in the language. I hated my algebra trig teacher in school and for some reason these functions have remained rather opaque to me. It would never have occurred to me during design that I could manipulate the density function by dividing by a number less than one before passing it to the softmax. Even after this discussion I still can’t do the math in my head, so I will combine this graph with the synthetic data suggestion and see if I can demonstrate explicitly in PyTorch.
Regards,
Kevin
who knew?