Tensorflow Slim debugging during training

Question

I am largely following this for the inception model. Let's say I want to print the image data being used in the batch on every training loop i.e. I want to print the value of 'images' variable for every training iteration in the 'Fine-tune the model on a different set of labels.' code, how do I do this since the session is actually being created inside the slim.training.train function hence I can't do a sess.run([images]) without sess here ?

Answer 1

Both precedent answers are very good to visualize images precisely.

However, if you are interested to print any other variable value or even the pixel intensities of the image, you can use the following code, right before calling slim.learning.train for instance :

    variable = tf.get_default_graph().get_tensor_by_name("variable_name")
    train_tensor = tf.Print(train_tensor, [variable], text_to_print_before_value, summarize=number_of_values_to_print_if_array)

The first line gets the variable by its name (you can get it via tensorboard or by printing slim.get_model_variables() for instance).

Second line prints it when train_tensor is evaluated and returns the train_tensor itself so that this Print node is added to the graph.

An example printing a variable from a Mobilenet model:

    beta = tf.get_default_graph().get_tensor_by_name("MobilenetV1/Conv2d_12_pointwise/BatchNorm/beta:0")
    train_tensor = tf.Print(train_tensor, [beta], "beta ", summarize=256)

Answer 2

So you want to visualize the training images? Well, you can add an image summary anywhere you have a handle to them, e.g. under "Train the model on the Flowers dataset.":

images, _, labels = load_batch(dataset)
tf.summary.image("input/images", images)

Answer 3

With the following script you can see your images: summaries.add(tf.summary.image('image_tensor,max_images=15,name='images'))

tf.summary.image is the easiest method because as mentioned session op is created inside slim.learning.train. Although interactivesession() can be used in the middle of the code, but it may disturb the session created by slim!