Save tensorflow object detection augmented images

Question

Is there a way to view the images that tensorflow object detection api trains on after all preprocessing/augmentation.

I'd like to verify that things look correctly. I was able to verify the resizing my looking at the graph post resize in inference but I obviously can't do that for augmentation options.

In the past with Keras I've been able to do that and I've found that I was to aggressive.

Answer 1

The API provides test code for augmentation options. In input_test.py file, the function test_apply_image_and_box_augmentation is for that. You can rewrite this function by passing your own images to the tensor_dict and then save the augmented_tensor_dict_out for verification or you can directly visualize it.

EDIT: Since this answer was long ago answered and still not accepted, I decided to provide a more specific answer with examples. I wrote a little test script called augmentation_test.py.

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import functools
import os
from absl.testing import parameterized

import numpy as np
import tensorflow as tf
from scipy.misc import imsave, imread

from object_detection import inputs
from object_detection.core import preprocessor
from object_detection.core import standard_fields as fields
from object_detection.utils import config_util
from object_detection.utils import test_case

FLAGS = tf.flags.FLAGS

class DataAugmentationFnTest(test_case.TestCase):

  def test_apply_image_and_box_augmentation(self):
    data_augmentation_options = [
        (preprocessor.random_horizontal_flip, {
        })
    ]
    data_augmentation_fn = functools.partial(
        inputs.augment_input_data,
        data_augmentation_options=data_augmentation_options)
    tensor_dict = {
        fields.InputDataFields.image:
            tf.constant(imread('lena.jpeg').astype(np.float32)),
        fields.InputDataFields.groundtruth_boxes:
            tf.constant(np.array([[.5, .5, 1., 1.]], np.float32))
    }
    augmented_tensor_dict = 
        data_augmentation_fn(tensor_dict=tensor_dict)
    with self.test_session() as sess:
      augmented_tensor_dict_out = sess.run(augmented_tensor_dict)
    imsave('lena_out.jpeg',augmented_tensor_dict_out[fields.InputDataFields.image])


if __name__ == '__main__':
  tf.test.main()

You can put this script under models/research/object_detection/ and simply run it with python augmentation_test.py. To successfully run it you should provide any image name 'lena.jpeg' and the output image after augmentation would be saved as 'lena_out.jpeg'.

I ran it with the 'lena' image and here is the result before augmentation and after augmentation.

.

Note that I used preprocessor.random_horizontal_flip in the script. And the result showed exactly what the input image looks like after random_horizontal_flip. To test it with other augmentation options, you can replace the random_horizontal_flip with other methods (which are all defined in preprocessor.py and also in the config proto file), all you can append other options to the data_augmentation_options list, for example:

data_augmentation_options = [(preprocessor.resize_image, {
        'new_height': 20,
        'new_width': 20,
        'method': tf.image.ResizeMethod.NEAREST_NEIGHBOR
    }),(preprocessor.random_horizontal_flip, {
    })]