I think this is one of big gaps between educational contents and real-world implementations.
Lucy dug a good point. A template for linear_activation_backward() only considers two strings for activations, i.e., “relu” and “sigmoid”. No “leaky relu”, “tanh”, “softmax” and so on. That’s absolutely OK. But, in the real world implementations, we usually add “otherwise/else” to avoid from falling into unpredictable conditions. I’m from the industry side, and feel that some programming examples are only focusing on features/functions that are being focused, and not caring about coding conventions and best practices. If this is a real world system, then, Lucy will get an error message to say “an activation function sigmod is not supported”, which definitely help Lucy’s debugging.