Week3___Tensorflow Introduction__Compute cost

[Jiajun]

I try to compute the cost, here is the code i wrote

cost = tf.reduce_mean(tf.math.maximum(logits,0)-logits*labels+tf.math.log(1+tf.math.exp(-abs(logits))),axis=-1)

However, i got this error,

The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

[vishnupriya755]

getting same error. Did u get the solution and can u pls help me with initilise parameters code, if possible

[kampamocha]

I think you should use the binary_crossentropy function as stated in section 3.2.

Please check if that helps.

[FatHaz]

Hi kampamocha,
I’m struggling with Exercise 6 - compute_cost. Can you give me any guidance in light of the following?

I can see that the lales are 1 or 0, and that the logits seem to be probabilities, hence from_logits=False, but something seems to be amiss.

Regards, Harry

[kampamocha]

Hi @FatHaz,

You need to compute the mean over the binary_crossentropy function, please check the example shown in the initial paragraph on section 3.2. Also note the value of from_logits parameter, since this is associated to y_pred rather than labels.

[FatHaz]

Hi kampamocha

Most obliged. I’d added the mean before I read your email, but the key was changing from False to True!

Thank you, Harry

[Fredy]

your were missing the other call of tf function as tf.keras.metrics.binary_crossentropy
by thw way, there was other ways i found can share as below,

cost = tf.reduce_mean(tf.reduce_mean(tf.maximum(logits, 0) - logits * labels + tf.math.log(1 + tf.math.exp(-abs(logits))), axis=-1))

[Saransh_Jhunjhunwala]

Can some please explain why the code in the paragraph above the cost exercise, that is -

cost = tf.reduce_mean(tf.keras.losses.binary_crossentropy(y_true = logits, y_pred = labels, from_logits=True))

is not working.

Whereas, -

cost = tf.reduce_mean(tf.reduce_mean(tf.maximum(logits, 0) - logits * labels + tf.math.log(1 + tf.math.exp(-abs(logits))), axis=-1))

Is working properly?

Please if someone can explain.

Thank you,
Saransh

[nramon]

Hi, @Saransh_Jhunjhunwala.

Make sure you understand the difference between y_true and y_pred and let me know if you need any hints.

Check this post if you’re not sure where the second formula comes from.

Good luck with the assignment

[Saransh_Jhunjhunwala]

@Saransh_Jhunjhunwala

cost = tf.reduce_mean(tf.keras.losses.binary_crossentropy(y_true = logits, y_pred = labels, from_logits=True))

Understood my mistake,
Thank you

[jinjh2018]

cost = tf.reduce_mean(tf.keras.losses.categorical_crossentropy(tf.transpose(labels),tf.transpose(logits),from_logits = True))
it works
and in your answer, notice that y true = labels

[Hussam]

I think you should use
tf.keras.metrics.categorical_crossentropy
instead of the binary.

[Hussam]

I think tf.keras.metrics.categorical_crossentropy is the correct implementation

[kampamocha]

Hi @Hussam,

You are right, we should use categorical_cross_entropy. At the time of my previous comment the notebook specified that you should use binary_cross_entropy, later that was revised and changed to what it is now, which makes more sense.

If you are interested, you can check some discussions about it:

(Tensorflow assignment )Why not compute A3 in forward_prop function?

That’s because they want you to use the Binary_crossentropy loss function with the from_logits = True argument, which causes the sigmoid calculation to be incorporated in the loss computation. That is preferred because they can manage numerical stability better when they do the two together. E.g. dealing with problems with “saturated” sigmoid output values. Here’s the doc page for Binary_crossentropy. They don’t really explain what the from_logits = True argument does in the assignment, but the…

Cannot compute_cost course 2 week 3

Update: Please note that after this thread was created, the Course Staff made some significant updates to this assignment, which include switching to using CategoricalCrossEntropy for the loss function in this section. Because we specify the from_logits = True argument, that means that the loss logic will apply either sigmoid or softmax to the logits to compute the actual \hat{y} values and then will compute the cross entropy loss between the predictions and the labels: -y_true * log(sigmoid(y…

[Suri_Mtui1]

@jinjh2018

tf.reduce_mean(tf.keras.losses.categorical_crossentropy(tf.transpose(labels),tf.transpose(logits),from_logits = True))

Why did we need to transpose the labels and logits?

[saifkhanengr]

Read this where Mentor Kin @Kic explains why we need to do that.

Best,
Saif.

[paulinpaloalto]

Also note that this is an old thread and refers to the way this assignment used to work. The transpose and from_logits parts are still valid, but they changed it a while ago so that we sum the loss values instead of computing the mean. Here’s a thread which explains why they made that change.