Improved implementation of softmax - Neural network training | Coursera

[gent.spah]

See if you searched a bit about it one of our mentors here @rmwkwok has written a great post about this, check it out:

Why is`from_logits=True` plus `activation=linear` more stable?

From C2 W2 “Improved implementation of softmax”, we know that, for a binary classification problem, the following approach A is more stable than the approach B: Approach Output layer’s activation Loss Function A “linear” tf.keras.losses.BinaryCrossentropy(from_logits=True) B “sigmoid” tf.keras.losses.BinaryCrossentropy(from_logits=False) This post will show the maths reason, and begin with the following slide: [image] The lecture replaces the middle equation (Approach B) with th…