Trigger word detection_ finding the Precision, F1 score and the confusion matrix of the model

Course Q&A Deep Learning Specialization Sequence Models
1

How to obtain the Precision, F1 score, and the confusion matrix of the model?

1 Like
2

Since you have the actual and predicted outputs, you can use sklearn.metrics.ConfusionMatrixDisplay.from_predictions to view the confusion matrix.

Here are some more functions from the same package:

  1. F1 score
  2. Recall
  3. Precision
  4. Confusion matrix
3

Thank you for your reply.
I am confused because when I am using the following code

from sklearn.metrics import classification_report, confusion_matrix
y_pred = model.predict(X_dev)
y_pred_classes = np.argmax(y_pred, axis=1)
y_true_classes = np.argmax(Y_dev, axis=1)
cm = confusion_matrix(y_true_classes, y_pred_classes)
print(“Confusion matrix:”)
print(cm)
print(“\nClassification report:”)
print(classification_report(y_true_classes, y_pred_classes))
print(“Confusion matrix:”)
print(cm)
print(“\nClassification report:”)
print(classification_report(y_true_classes, y_pred_classes))

I am getting the following results

F1 = prescision = accuracy = 0

is the problem with the written code? can I obtain them after using the detect_triggerword function? Can you give me more details about using them in this code?

4

Please use the snippet below:

from sklearn.metrics import classification_report, confusion_matrix
y_pred = model.predict(X_dev)
y_pred_classes = y_pred.round() # make the values equal to classes i.e. 0/1
y_pred_classes = y_pred_classes.reshape((-1, 1)) # concatenate all predictions
y_true_classes = Y_dev.reshape((-1, 1))
cm = confusion_matrix(y_true_classes, y_pred_classes)
print(“Confusion matrix:”)
print(cm)
print(“\nClassification report:”)
print(classification_report(y_true_classes, y_pred_classes))
print(“Confusion matrix:”)
print(cm)
print(“\nClassification report:”)
print(classification_report(y_true_classes, y_pred_classes))
5

ValueError Traceback (most recent call last)
in
4 y_pred_classes = y_pred.reshape((-1, 1)) # concatenate all predictions
5 y_true_classes = Y_dev.reshape((-1, 1))
----> 6 cm = confusion_matrix(y_true_classes, y_pred_classes)
7 print(“Confusion matrix:”)
8 print(cm)

/opt/conda/lib/python3.7/site-packages/sklearn/metrics/_classification.py in confusion_matrix(y_true, y_pred, labels, sample_weight, normalize)
266
267 “”"
→ 268 y_type, y_true, y_pred = _check_targets(y_true, y_pred)
269 if y_type not in (“binary”, “multiclass”):
270 raise ValueError("s is not supported" y_type)

/opt/conda/lib/python3.7/site-packages/sklearn/metrics/_classification.py in _check_targets(y_true, y_pred)
88 if len(y_type) > 1:
89 raise ValueError("Classification metrics can’t handle a mix of {0} "
—> 90 “and {1} targets”.format(type_true, type_pred))
91
92 # We can’t have more than one value on y_type => The set is no more needed

ValueError: Classification metrics can’t handle a mix of binary and continuous targets

it is giving this error

6

Sorry about that. Made a typo in my previous reply. Fixed it now.

7

It works :smile:
thank you very much, I really appreciate your kind help :smile: