Graph as below:
Drawing this graph really helps me digest all kinds of stuffs around a NN into a fluent flow. Hope it help you too, and discussion is warmly welcomed.