Question about is_logit

[rmwkwok]

@Khalid_A.W

However, I am struggling to understand why it is “preferable” in this case to use linear instead of sigmoid for the activation in a binary classification problem.

Hello @Khalid_A.W,

We don’t just prefer to use linear instead of sigmoid for activation. We prefer to use linear for activation AND setting from_logit to True in the loss function that’s passed into the model training. If you following these steps, you will see that sigmoid is never out of the game.

Again, sigmoid is there if we set from_logit to True. It is NOT in the output layer because we have linear for the layer’s activation, however, it IS in the loss function if we set from_logit to True.

@Khalid_A.W

Also, to make sure I am understanding this implementation correctly, do we get the same result whether we use “from_logits = True” as manually changing the activation of the output layer from “softmax” to “linear” (without adding the “from_logits = True” argument to the loss function)?

Do the experiment yourself

Cheers,
Raymond