dA0 informs us how we can increase the cost w.r.t. the inputs. This essentially provides us a direction from the current training example in which we expect a high cost. Can this be used to come up with adversarial examples? Or even better can this inform us about which training example can be used in the next training iteration?
Is there any ongoing research in this area? A list of papers would be very helpful. Thanks!
It’s interesting to consider whether there is any possible use for the dA0 information. It’s generated as a side effect of the way back prop works. I think Prof Ng comments at some point that we just discard this information since we can’t modify the inputs. Given that we can’t modify the inputs, is there anything useful we can do with that information? Note that dA0 will be a matrix of the same shape as X. Looking at the “columnwise” norms might tell you which samples are having more influence on the behavior. Maybe you could use that information to make your training more efficient by subsetting your training samples to some subset that seems to be the most infuential. Or at least eliminate those that don’t seem to be having any effect. Or maybe you could find cases in which two samples cancel each other out.
I’m just “free associating” here, so these ideas are probably worth exactly what you paid for them. 
I have not looked into the matter and have never heard it discussed before, but you can try doing some googling to see you can find anything about this in the literature. Let us know if you find anything of note!