Keras: Validation error is a good measure for stop

2019-08-16 05:38发布

问题:

I am using Keras for multi-class classification problem. I am using EarlyStopping(monitor='val_loss', patience=4) as stopping criteria for learning, that is if validation loss does not decrease for 4 epochs, training stops. Is it better to use val_acc as stopping criteria or val_loss? Because I see val_loss increasing but also val_acc. Considering following output of Epoch 8 and Epoch 13.

Epoch 1/200
240703/240703 [==============================] - 4831s - loss: 0.8581 - acc: 0.7603 - val_loss: 0.6247 - val_acc: 0.8160
Epoch 2/200
240703/240703 [==============================] - 4855s - loss: 0.6099 - acc: 0.8166 - val_loss: 0.5742 - val_acc: 0.8300
Epoch 3/200
240703/240703 [==============================] - 4627s - loss: 0.5573 - acc: 0.8308 - val_loss: 0.5600 - val_acc: 0.8337
Epoch 4/200
240703/240703 [==============================] - 4624s - loss: 0.5265 - acc: 0.8395 - val_loss: 0.5550 - val_acc: 0.8347
Epoch 5/200
240703/240703 [==============================] - 4623s - loss: 0.5042 - acc: 0.8452 - val_loss: 0.5529 - val_acc: 0.8377
Epoch 6/200
240703/240703 [==============================] - 4624s - loss: 0.4879 - acc: 0.8507 - val_loss: 0.5521 - val_acc: 0.8378
Epoch 7/200
240703/240703 [==============================] - 4625s - loss: 0.4726 - acc: 0.8555 - val_loss: 0.5554 - val_acc: 0.8383
Epoch 8/200
240703/240703 [==============================] - 4621s - loss: 0.4604 - acc: 0.8585 - val_loss: 0.5513 - val_acc: 0.8383
Epoch 9/200
240703/240703 [==============================] - 4716s - loss: 0.4508 - acc: 0.8606 - val_loss: 0.5649 - val_acc: 0.8366
Epoch 10/200
240703/240703 [==============================] - 4602s - loss: 0.4409 - acc: 0.8637 - val_loss: 0.5626 - val_acc: 0.8389
Epoch 11/200
240703/240703 [==============================] - 4651s - loss: 0.4318 - acc: 0.8662 - val_loss: 0.5710 - val_acc: 0.8387
Epoch 12/200
240703/240703 [==============================] - 4706s - loss: 0.4239 - acc: 0.8687 - val_loss: 0.5737 - val_acc: 0.8384
Epoch 13/200
240703/240703 [==============================] - 4706s - loss: 0.4190 - acc: 0.8698 - val_loss: 0.5730 - val_acc: 0.8391

回答1:

In general loss is a better measure than accuracy because it has higher precision. Accuracy has as many possible values as many samples you have in the validation set. Loss, on the other hand has continuum possible values so you can track what is happening with better precision. On the other hand, accuracy is easier to analyse since it is interpretable (it is just a percentage), so without an expertise in the field - loss based criterion will be harder to use, but probably slightly more precise (if used correctly).