C3W3 Assignment Duplicates Exercise 4 Evaluation/classify - Batch Loop & Confusion matrix

Course Q&A Natural Language Processing NLP with Sequence Models
1

I am having a hard time solving Exercise 4 based on the hints given and the structure of the program. I am feeling that I am making it much more complex than it should be, such as creating additional rows or variables that may not be recognized by the automatic correction
labID pxpenjemklvz
location:/notebooks/C3W3_Assignment.ipynb

  1. do we need to still loop thru the data set and match y_test - or we create a dataiterator for y_test ?
    something like
    for i in range(0, len(test_Q1), batch_size):
    pred = model.predict(next(iter(test_gen))[0], …)
    y_label = tf.cast( y_test[i:i + batch_size],tf.float64)

  2. do we compute accuracy in incremental for every batch
    accuracy = 0 before the loop
    accuracy += blabla in each loop

  3. do we do the same (sum thru batches) for the confusion matrix or is it a way to call it on the global test set after the loop
    do I need to create a y_true for the confusion matrix
    y_true = tf.cast((y_label == y_pred), tf.float64)

2

Hi @Fred_Hannoyer

We don’t need the for loop in this case - model.predict is enough. Please check these previous answers:

Cheers

3

Dear Arvyzukai

I am sorry but the previous answers did not help me, except confirm me that it should be simpler than my solution (=less lines) and answered only one of my questions 1) without telling me whether the form was correct.

Would you please look at my notebook ? and maybe point me to the previous exercise where we did such global loop on tf.dataset ?

4

Hi @Fred_Hannoyer

Forgive me for being concise as I’m short on time right now. I cannot look at your code right now, maybe later.

  1. we don’t need the for loop, we call model.predict on test_gen. Result of this is predictions for the whole test dataset.
  2. no (see answer 1) we compare all predictions for the whole dataset (y_test and y_pred are 10240 long)
  3. no (see answer 1), use last instruction

Cheers

5

Follow exactly your path that is the most intuitive - this is executing, predict outputs 10240 vectors and generate all computes on the global set seamlessly of the batch slicing
it worked except I got wrong results at the end for accuracy and confusion

personnally my only doubts are

  • you do cosim as v2*v1.T using tf.math.reduce_sum(tf.linalg.matmul(… transpose_b=True),axis=1)
  • you sum the accuracy (d>threshold) /total_inputset only if y_pred==y_test you use a cast-float32 for the mask with tf.math.reduce_sum(y_pred*tf.cast(y_test == y_pred,tf.float64))/total_inputset

image
image732×309 7.66 KB

PS: i am trying to be unambiguous for other students, without putting the code verbatim, and but if there are too much code I can remove it

6

In this case you’re asked to use tf.math.reduce_sum(), which is equivalent to what you have with tf.linalg.matmul. So, you should use only one of them, not both.

No need to multiply by y_pred, it doesn’t make sense.

Cheers

1 Like