Yes, it seems to make a big difference. 
Convolutions are assisting the Network in identifying the edges versus having the network learn some other patterns that may or may not be relevant to the classification. Thank you for sharing.
From Week 1 of Structuring Machine Learning Course- We can see that Variance ( difference in Dev error and Train error) is roughly 5%, Avoidable bias (difference between training error and human error) is roughly 16%.
I was playing around with another CNN lab on a different dataset. I was able to reduce the variance just by increasing the number of filters. I think you definitely have room for improvement. e.g hyper parameter tuning, etc.
Hi all,
just completed the training of Resnet 50 on kaggle cats vs dogs dataset, training without any pre-trained weights.
it is one step further than my 2 layers CNN used on the previous post.
here is the result:
Epoch 1 - Train Loss: 0.2472520781, Accuracy: 89.5750%, Test Loss: 0.1684890389, Test Accuracy: 93.1400%
Epoch 10 - Train Loss: 0.0177410016, Accuracy: 99.3400%, Test Loss: 0.2259603292, Test Accuracy: 93.9800%
Epoch 20 - Train Loss: 0.0119494755, Accuracy: 99.5550%, Test Loss: 0.2812386155, Test Accuracy: 93.5200%
Epoch 30 - Train Loss: 0.0056119472, Accuracy: 99.8150%, Test Loss: 0.2672079206, Test Accuracy: 93.6200%
Epoch 40 - Train Loss: 0.0085476152, Accuracy: 99.7050%, Test Loss: 0.2651146352, Test Accuracy: 93.8800%
Epoch 50 - Train Loss: 0.0081489242, Accuracy: 99.7600%, Test Loss: 0.2696788013, Test Accuracy: 94.0800%
Epoch 60 - Train Loss: 0.0060868834, Accuracy: 99.8300%, Test Loss: 0.2607414126, Test Accuracy: 94.2400%
Epoch 70 - Train Loss: 0.0019748662, Accuracy: 99.9350%, Test Loss: 0.4114777744, Test Accuracy: 93.5800%
Epoch 80 - Train Loss: 0.0000091611, Accuracy: 100.0000%, Test Loss: 0.3825938404, Test Accuracy: 94.5800%
Epoch 90 - Train Loss: 0.0000006609, Accuracy: 100.0000%, Test Loss: 0.4616193473, Test Accuracy: 94.5800%
Epoch 100 - Train Loss: 0.0000001236, Accuracy: 100.0000%, Test Loss: 0.5290119052, Test Accuracy: 94.4400%
Epoch 110 - Train Loss: 0.0033128520, Accuracy: 99.9050%, Test Loss: 0.3774854243, Test Accuracy: 94.0600%
Epoch 120 - Train Loss: 0.0032119655, Accuracy: 99.9100%, Test Loss: 0.3305169046, Test Accuracy: 94.1600%
Epoch 130 - Train Loss: 0.0015249494, Accuracy: 99.9400%, Test Loss: 0.3866370618, Test Accuracy: 93.5800%
Epoch 140 - Train Loss: 0.0041348637, Accuracy: 99.8650%, Test Loss: 0.2674987018, Test Accuracy: 94.1400%
Epoch 150 - Train Loss: 0.0058022594, Accuracy: 99.8250%, Test Loss: 0.3452861309, Test Accuracy: 93.2000%
Epoch 160 - Train Loss: 0.0000019746, Accuracy: 100.0000%, Test Loss: 0.4157975614, Test Accuracy: 94.0800%
Epoch 170 - Train Loss: 0.0009905830, Accuracy: 99.9700%, Test Loss: 0.3247412145, Test Accuracy: 94.3000%
Epoch 180 - Train Loss: 0.0029946355, Accuracy: 99.9300%, Test Loss: 0.3015829027, Test Accuracy: 94.1600%
Epoch 190 - Train Loss: 0.0000016242, Accuracy: 100.0000%, Test Loss: 0.4693199098, Test Accuracy: 94.2000%
Epoch 200 - Train Loss: 0.0000001741, Accuracy: 100.0000%, Test Loss: 0.5655826926, Test Accuracy: 94.0800%
Epoch 210 - Train Loss: 0.0000000757, Accuracy: 100.0000%, Test Loss: 0.6774858832, Test Accuracy: 94.1600%
Epoch 220 - Train Loss: 0.0007556589, Accuracy: 99.9800%, Test Loss: 0.4170395136, Test Accuracy: 93.6600%
Epoch 230 - Train Loss: 0.0004256574, Accuracy: 99.9950%, Test Loss: 0.4657711983, Test Accuracy: 93.6200%
Epoch 240 - Train Loss: 0.0000022613, Accuracy: 100.0000%, Test Loss: 0.4250835478, Test Accuracy: 94.1200%
Epoch 250 - Train Loss: 0.0000003651, Accuracy: 100.0000%, Test Loss: 0.5500867963, Test Accuracy: 94.2400%
Epoch 260 - Train Loss: 0.0000028924, Accuracy: 100.0000%, Test Loss: 0.4785893559, Test Accuracy: 93.9400%
Epoch 270 - Train Loss: 0.0000004237, Accuracy: 100.0000%, Test Loss: 0.5304982066, Test Accuracy: 94.0200%
Epoch 280 - Train Loss: 0.0000000673, Accuracy: 100.0000%, Test Loss: 0.6908071637, Test Accuracy: 94.1400%
Epoch 290 - Train Loss: 0.0000000124, Accuracy: 100.0000%, Test Loss: 0.7716891766, Test Accuracy: 94.2000%
Epoch 300 - Train Loss: 0.0000383568, Accuracy: 100.0000%, Test Loss: 0.3829227388, Test Accuracy: 94.1200%
Finished Training. Total training time: 1106.41 minutes
Training Accuracy: 100.00%
Testing Accuracy: 94.10%
here are the curves:
accuracy graph:
loss curve:
the accuracy can be as high as 94%
(really long way for me starting from fully connected Neural Networks with only 70% accuracy)
now it is definitely overfitting.
but comparing to the best 98% which is not a very big difference.
I will try a few more CNN models e.g. VGG16, AlexNet, InceptionV3, MobileNet v1, all mentioned in Course 4.
and will give the results later.
it probably would take long time to have result.
ResNet50 took me 1106min to train which is > 18hr.
