I have trained an object detection model. Now I'm trying to speed it up for inference using quantization provided by Tensorflow Lite graph converter. But when I call the tf.lite.TFLiteConverter.from_frozen_graph
method, I am running into an error.
I have also found a similar, unanswered question asked almost a year ago and I was wondering if TFLite's support has improved now.
Here is what I'm calling:
converter = tf.lite.TFLiteConverter.from_frozen_graph(
model_path,
input_arrays = ['input_1'],
output_arrays = [
'filtered_detections/map/TensorArrayStack/TensorArrayGatherV3',
'filtered_detections/map/TensorArrayStack_1/TensorArrayGatherV3',
'filtered_detections/map/TensorArrayStack_2/TensorArrayGatherV3'
],
input_shapes = {
'input_1': [None, 300, 300, 3]
}
)
converter.post_training_quantize = True
tflite_quantized_model = converter.convert()
EDIT: I have also tried different parameter values for input_1 like [1, 300, 300, 3] etc. I have even left out the input_shapes parameter but then it throws another error: None is allowed only in 1st dimension. Other dimensions can not be null
Here are the error logs:
File "lib/python3.6/site-packages/tensorflow/lite/python/lite.py", line 500, in convert
**converter_kwargs)
File "lib/python3.6/site-packages/tensorflow/lite/python/convert.py", line 442, in toco_convert_impl
input_data.SerializeToString())
File "lib/python3.6/site-packages/tensorflow/lite/python/convert.py", line 205, in toco_convert_protos
"TOCO failed. See console for info.\n%s\n%s\n" % (stdout, stderr))
tensorflow.lite.python.convert.ConverterError: TOCO failed. See console for info.
2019-02-06 18:38:40.906888: I tensorflow/lite/toco/import_tensorflow.cc:1332] Converting unsupported operation: Enter
2019-02-06 18:38:40.915666: I tensorflow/lite/toco/import_tensorflow.cc:1332] Converting unsupported operation: Enter
2019-02-06 18:38:40.917286: I tensorflow/lite/toco/import_tensorflow.cc:1332] Converting unsupported operation: Size
2019-02-06 18:38:40.917308: I tensorflow/lite/toco/import_tensorflow.cc:1332] Converting unsupported operation: Size
...
...
2019-02-06 18:38:40.918758: I tensorflow/lite/toco/import_tensorflow.cc:1332] Converting unsupported operation: TensorArrayWriteV3
2019-02-06 18:38:40.918783: I tensorflow/lite/toco/import_tensorflow.cc:1332] Converting unsupported operation: TensorArrayWriteV3
2019-02-06 18:38:40.918796: I tensorflow/lite/toco/import_tensorflow.cc:1332] Converting unsupported operation: TensorArrayWriteV3
2019-02-06 18:38:40.935936: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] Before Removing unused ops: 952 operators, 1408 arrays (0 quantized)
2019-02-06 18:38:40.955338: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] Before general graph transformations: 952 operators, 1408 arrays (0 quantized)
2019-02-06 18:38:41.234167: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] After general graph transformations pass 1: 396 operators, 708 arrays (0 quantized)
2019-02-06 18:38:41.242773: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] Before Group bidirectional sequence lstm/rnn: 396 operators, 708 arrays (0 quantized)
2019-02-06 18:38:41.249476: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] Before dequantization graph transformations: 396 operators, 708 arrays (0 quantized)
2019-02-06 18:38:41.262130: F tensorflow/lite/toco/tooling_util.cc:633] Check failed: dim >= 1 (0 vs. 1)
Fatal Python error: Aborted
Current thread 0x00007f4930238740 (most recent call first):
File "lib/python3.6/site-packages/tensorflow/lite/toco/python/toco_from_protos.py", line 33 in execute
File "lib/python3.6/site-packages/absl/app.py", line 251 in _run_main
File "lib/python3.6/site-packages/absl/app.py", line 300 in run
File "lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 40 in run
File "lib/python3.6/site-packages/tensorflow/lite/toco/python/toco_from_protos.py", line 59 in main
File "bin/toco_from_protos", line 11 in <module>
Aborted (core dumped)
