Keras cifar10 example validation and test loss low

Question

I'm playing around with the cifar10 example from Keras which you can find here. I've recreated the model (i.e., not same file but everything else pretty much the same) and you can find it here.

The model is identical and I train the model for 30 epochs with 0.2 validation split on the 50,000 image training set. I'm not able to understand the result I get. My validation and testing loss is lesser than the training less (inversely, training accuracy is the lower compared to the validation and testing accuracy):

                      Loss       Accuracy
   Training          1.345          0.572
 Validation          1.184          0.596
       Test           1.19          0.596

Looking at the plot, I'm not sure why the training error starts increasing again so badly. Do I need to reduce the number of epochs I train for or maybe implement early stopping? Would a different model architecture help? If so, what would be good suggestions?

Thanks.

Answer 1

This is a rare phenomenon but it happens from time to time. There are several reasons why this might be the case:

• smaller datasets have smaller intrinsic variance so this means that your model properly captures patterns inside of your data and train error is greater simply because the inner variance of training set is greater then validation set.
• Simple accident - this might also occur - that your split is suitable for such behaviour.