Assignment 2 - Dense Layer Activation

It’s a little more subtle than that: we actually are using a sigmoid activation at the output layer, but we do it as part of the loss function. Note that we use the from_logits = True argument to the cross entropy loss function.

Here’s a thread which explains why we do it that way. It will always work this way and we first encountered this back in DLS C2 W3 in the TensorFlow Introduction assignment, which is what that thread is discussing.

Then the additional point is that when you want to use the trained model to make a prediction, you either manually add the sigmoid or change the interpretation of the output to say > 0 means True.

1 Like