So, I can see that the diagram can be divided into top / middle / bottom parts.
Arrows go from left to right in the top part, showing the forward prop. Arrows, on the other hand, go from right to left in the bottom part for back prop.
In the middle part, downward arrows show values are cached from the top and then used for the bottom. Upward arrows show the updates.
I think the diagram might be slightly simplified by skipping the operation names such as 
, because, for example, you would just have 
directly instead of adding names in between.
Also, I mentioned 4 kinds of arrows above, so I wonder if it would be interesting to you to see different colors/types of arrows for them.
There is just one additional type of relation you might add to the diagram. If you watch the computational graph video again (this DLS video or this MLS video), you see that each math operation in the forward prop is related to a derivative operation in the back prop. For example,
However, I am not sure if it is better to add these relations as it will make the diagram more complex.
Lastly, in my first two jobs, I used AutoCAD and SolidWork a lot, so at that time I read many things about how to draw. Your diagram reminded me of how lines can be jumped when they cross the others. Check this out for an example. This is particularly useful when all arrows have the same color and style. I mean, if you like it and your diagram tool supports it, then why not?
Cheers,
Raymond