In the alpaca_model (Exercise 2), we imported the MobileNetV2 model (without the top layer) and added new layers for our binary classification task. When we train the model for 5 epochs right after, the accuracy is decent but could be better. Here is a screenshot of my accuracies around 75%. However, I am very concerned about whether they are correct!
If I take 32 training examples and put them in the model, the model seems to almost always output a number between 1 and 1.5, regardless of whether the image is an alpaca or not. See screenshot below. I added those cells immediately after training.
If I am understanding this correctly, since we only have one output and use binary labeling, the output means alpaca if it is closer to 0, and means not alpaca if it is closer to 1? If that’s true, the model doesn’t seem to be able to recognize alpacas at all. I am not sure how the training and validation accuracies are calculated exactly, but they don’t seem to be reflecting what’s actually happening here. I found some discussions about this.
- List item machine learning - ResNet: 100% accuracy during training, but 33% prediction accuracy with the same data - Stack Overflow The accepted answer suggested that the issue is with BatchNorm and that we change momentum, but I don’t quite understand why it should help. After trying it myself, I don’t think it worked for me.
It also concerns me that the validation accuracy is usually higher than training accuracy. I don’t think that’s supposed to happen so often either?
PS. I already finished this course and got 100/100 on this assignment. I am asking this because I am trying to imitate the assignment and build a classifier with 3 classes (by changing to categorical label and categorical cross entropy) using my own training data. I noticed that when doing model.fit(…) myself, the accuracies are very high (>95%), but when tested on individual training examples, the model does not output the correct prediction.
I know there are many black boxes I don’t yet understand. I would really appreciate it if anyone is able to help!