How to create a loss function which changes over e

Question

I would like to create a custom loss function that has a weight term that's updated based on what epoch I'm in.

For example: Let's say I have a loss function which has a beta weight, where beta increases over the first 20 epochs...

def custom_loss(x, x_pred): 
    loss1 = objectives.binary_crossentropy(x, x_pred)
    loss2 = objectives.mse(x, x_pred)
    return (beta*current_epoch/20) * loss1 + loss2

How could I implement something like this into a keras loss function?

Answer 1

Looking at their documentation they mention that you can use theano/Tf symbolic functions that return a scalar for each data point. So you could do something like this

loss = tf.contrib.losses.softmax_cross_entropy(x, x_pred) * 
       (beta * current_epoch / 20 ) +  
       tf.contrib.losses.mean_squared_error

You would have to pass x and x_pred as x and x_pred as tf.placeholders I think for model creation you could use keras but then again you would have to run the computational graph with sess.run()

References: https://blog.keras.io/keras-as-a-simplified-interface-to-tensorflow-tutorial.html#using-keras-models-with-tensorflow