Unable to understand tensorflow class assigment error

hi everybody i am doing the programming exercise about tensorflow (Tensorflow_introduction) but i am blocked on this function:

def compute_total_loss(logits, labels)

i am computing cost function in the following manner but it seems there is an error when i run the test compute_total_loss_test(target, Y):
the test calculates 0,81028707
my function 0.8071431

my cost function is the following:

total_loss = tf.reduce_sum(tf.keras.losses.categorical_crossentropy(tf.transpose(labels), tf.transpose(logits)))/logits.shape[1]

may someone help me to understand the error?


Hi, here is a thread about this question that might be helpful.

denghiu now i will give a lookk

my function seems correct but results is not as expected by the test function

Note that the cost here is defined as the total sum of the loss values, not the mean, as it is in some other places. In the first code you showed, you are manually dividing by the number of samples. If you wanted the mean there is also a reduce_mean function in TF which would make that easier, but the real point is that is not what they are asking us to do here.

The other thing that was missing from your original code shown above is that you need to use the from_logits argument to tell the loss function to apply the softmax activation internally.

ok thank you i will try but isn’t specified anyway i have to use softmax

Well, maybe they don’t explicitly say that, but the lectures have been talking about that, right? The test case is doing a multiclass classification with 6 classes, right? So sigmoid is not going to work and softmax is your only hope. Notice that the forward propagation function that you defined earlier does not apply an activation function at the output layer, but just gives the linear outputs (“logits”).

That is the way Prof Ng always does it once we switch to using TensorFlow: it is better to let the loss function include the activation calculation internally as part of the loss calculation.

ok if i use from_logit=True and i drop the division by numner_of_examples(/logits.shape[1]) is working … it was this sentence:
“you can sum the losses across many batches, and divide the sum by the total number of samples to get the cost value.” that makes me drop a mistake however thank you for your precious help :slight_smile:

1 Like

Yes, but what they are explaining there is that this cost function is just intended for each individual minibatch and then you compute the mean once you have completed the Epoch and have the total sum.

1 Like

yes now is clear thank you