More complex model causes decreased accuracy?

Course Q&A TensorFlow Developer Professional Convolutional Neural Networks in TensorFlow
1

I passed this assignment using 16 filters for the first conv2D layer, 64 filters in the second conv2D layer, and 64 units in the first Dense layer, which usually got accuracy between .32 and .45.

The weird thing (to me) is that when I tried to boost accuracy by adding more conv filters, more dense layers, or more units in the first dense layer, accuracy would drop to around 0.04. I have experimented with adding and removing dropout layers, but they didn’t help.

I can’t think what would cause this drop. Does anyone have insights? To anyone getting training accuracy ~0.8 or more, how did you do it?

P.S. I also experimented with making the softmax layer length 24 (number of unique training samples), 25, and 26 (length of the alphabet), but I didn’t notice an effect from that.

2

Hi Rachel,
I got the chance to tweak a little on your code. Instead than adding more conv layers you could try to:

  • increase the units in the first Dense layer
  • change the optimizer - I often start with ‘adam’, no learning rate
  • increase the epochs
  • change the batch_size to a more customary 32
  • I actually used Dropout too, but that comes later

Doing this, I’ve been able to increase your numbers.

Happy fine-tuning

2 Likes
3

Thanks for the encouragement and the suggestions, particularly the suggestion to use ‘adam’ with no learning rate. With 1024 units in the first dense layer, that switch alone brings the accuracy from ~0.045 to ~0.6!

Rachel

1 Like