gcloud ml-engine returns error on large files

Question

I have a trained model that takes in a somewhat large input. I generally do this as a numpy array of the shape (1,473,473,3). When I put that to JSON I end up getting about a 9.2MB file. Even if I convert that to a base64 encoding for the JSON file the input is still rather large.

ml-engine predict rejects my request when sending the JSON file with the following error:

(gcloud.ml-engine.predict) HTTP request failed. Response: {
"error": {
    "code": 400,
    "message": "Request payload size exceeds the limit: 1572864 bytes.",
    "status": "INVALID_ARGUMENT"
  }
}

It looks like I can't send anything over about 1.5MB in size to ML-engine. Is this for sure a thing? How do others get around doing online predictions for large data? Do I have to spin up a compute-engine or will I run into the same issue there?

Edit:

I am starting from a Keras model and trying to export to tensorflow serving. I load my Keras model into a variable named 'model' and have a defined directory "export_path". I build the tensorflow serving model like this:

signature = predict_signature_def(inputs={'input': model.input},
                                outputs={'output': model.output})
builder = saved_model_builder.SavedModelBuilder(export_path)
builder.add_meta_graph_and_variables(
    sess=sess,
    tags=[tag_constants.SERVING],
    signature_def_map={
        signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY: signature
    }
)
builder.save()

How would the input look for this signature_def? Would the JSON just be something like {'input': 'https://storage.googleapis.com/projectid/bucket/filename'} where the file is the (1,473,473,3) numpy array?

2nd Edit: Looking at the code posted by Lak Lakshmanan, I have tried a few different variations without success to read an image url and attempt to parse the file that way. I have tried the following without success:

inputs = {'imageurl': tf.placeholder(tf.string, shape=[None])}
filename = tf.squeeze(inputs['imageurl']) 
image = read_and_preprocess(filename)#custom preprocessing function
image = tf.placeholder_with_default(image, shape=[None, HEIGHT, WIDTH, NUM_CHANNELS])
features = {'image' : image}
inputs.update(features)
signature = predict_signature_def(inputs= inputs,
                                outputs={'output': model.output})


with K.get_session() as session:
    """Convert the Keras HDF5 model into TensorFlow SavedModel."""
    builder = saved_model_builder.SavedModelBuilder(export_path)
    builder.add_meta_graph_and_variables(
        sess=session,
        tags=[tag_constants.SERVING],
        signature_def_map={
            signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY: signature
        }
    )
    builder.save()

I believe the problem is with getting a mapping from the imageurl placeholder towards building the features. Thoughts on what I am doing wrong?

Answer 1

What I typically do is to have the json refer to a file in Google Cloud Storage. See the serving input function here for example:

https://github.com/GoogleCloudPlatform/training-data-analyst/blob/61ab2e175a629a968024a5d09e9f4666126f4894/courses/machine_learning/deepdive/08_image/flowersmodel/trainer/model.py#L119

Users would first upload their file to gcs and then invoke prediction. But this approach has other advantages, since the storage utilities allow for parallel and multithreaded uploads.