Cannot compute_cost course 2 week 3

Course Q&A Deep Learning Specialization Improving Deep Neural Networks: Hyperparameter tun
9

@paulinpaloalto , I did some more trials, the result shows as below:

trial
trial1938×730 244 KB

Note: Model trained with training set from Course 2 Week 3 assignment

Accordingly, it concludes that:

  1. For binary classification issue, from_logits=True without extra activation function works as good as sigmoid(logits) with binary_crossentropy loss function.
  2. For binary classification issue, both softmax and categorical_crossentropy loss function do not perform well.

key code as below:
Trial No. 1:

  1. cost = tf.reduce_mean(tf.keras.losses.binary_crossentropy(tf.transpose(labels), tf.transpose(logits), from_logits=True))

  2. y_pred = tf.keras.activations.sigmoid(logits)
    cost = tf.reduce_mean(tf.keras.losses.binary_crossentropy(tf.transpose(labels), tf.transpose(y_pred), from_logits=False))

Trial No. 2:

  1. cost = tf.reduce_mean(tf.keras.losses.categorical_crossentropy(tf.transpose(labels), tf.transpose(logits), from_logits=True))

  2. y_pred = tf.keras.activations.sigmoid(logits)
    cost = tf.reduce_mean(tf.keras.losses.categorical_crossentropy(tf.transpose(labels), tf.transpose(y_pred), from_logits=False))

Trial No. 3:

  1. cost = tf.reduce_mean(tf.keras.losses.binary_crossentropy(tf.transpose(labels), tf.transpose(logits), from_logits=True))

  2. cost = tf.reduce_mean(tf.keras.losses.categorical_crossentropy(tf.transpose(labels), tf.transpose(logits), from_logits=True))

Trial No. 4:

  1. y_pred = tf.keras.activations.softmax(logits)
    cost = tf.reduce_mean(tf.keras.losses.binary_crossentropy(tf.transpose(labels), tf.transpose(y_pred), from_logits=False))

  2. y_pred = tf.keras.activations.softmax(logits)
    cost = tf.reduce_mean(tf.keras.losses.categorical_crossentropy(tf.transpose(labels), tf.transpose(y_pred), from_logits=False))

13 Likes