Running trained tensorflow model in C++

Question

I have trained an image classification network in python using tensorflow. The trained model was saved as a .pb. Now, I want to test the model, I need this to be done in C++.

I had used numpy in manipulating and handling data. During training phase the image is passed in as a numpy array. The image is stretched out as a 1D array and the class label is prepended to this array.

I'm confused as to how to pass the image data while running the model in C++, where numpy isn't available to me. I use numpy operations to manipulate and handle the data. In what format should I pass in the data if I have to execute it in C++.

Below is how I train and save my model

def trainModel(data):
    global_step = tf.Variable(0, name='global_step', trainable=False)
    X, y,keep_prob = modelInputs((741, 620, 1),4)
    logits = cnnModel(X,keep_prob)
    cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=y), name="cost")
    optimizer = tf.train.AdamOptimizer(.0001, name='Adam').minimize(cost)
    prediction = tf.argmax(logits, 1, name="prediction")
    correct_pred = tf.equal(prediction, tf.argmax(y, 1), name="correct_pred")
    accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32), name='accuracy')
    with tf.Session() as sess:
        sess.run(tf.global_variables_initializer())
        saver = tf.train.Saver()
        batch_size = 30
        for e in range(11):
            batch_x, batch_y = data.next_batch(batch_size)
            batch_y = batch_y.astype('int32')
            x = np.reshape(batch_x, [batch_size, 741, 620, 1])
            labels = np.zeros(shape=(batch_size,4))
            labels[np.arange(len(labels)),batch_y]=1
            sess.run(optimizer, feed_dict={X: x, y: labels,keep_prob:0.5})
            global_step.assign(e).eval()
        saver.save(sess, './my_test_model',global_step=global_step)

*741x620 is the size of the image

Answer 1

Instructions for using a graph in C++ can be found here.

Here is some code to use your image as input:

tensorflow::Tensor keep_prob = tensorflow::Tensor(tensorflow::DT_FLOAT, tensorflow::TensorShape());
keep_prob.scalar<float>()() = 1.0;

tensorflow::Tensor input_tensor(tensorflow::DT_FLOAT, tensorflow::TensorShape({1,height,width,depth}));
auto input_tensor_mapped = input_tensor.tensor<float, 4>();
const float * source_data = (float*) img.data;  // here img is an opencv image, but if it's just a float array this code is very easy to adapt
// copying the image data into the corresponding tensor
for (int y = 0; y < height; ++y) {
    const float* source_row = source_data + (y * width * depth);
    for (int x = 0; x < width; ++x) {
        const float* source_pixel = source_row + (x * depth);
        for (int c = 0; c < depth; ++c) {
            const float* source_value = source_pixel + c;
            input_tensor_mapped(0, y, x, c) = *source_value;
        }
    }
}
std::vector<tensorflow::Tensor> finalOutput;

tensorflow::Status run_status = this->tf_session->Run({{InputName,input_tensor}, 
                                                       {dropoutPlaceHolderName, keep_prob}},
                                                      {OutputName},
                                                      {},
                                                      &finalOutput);