TensorFlow Lite quantization fails to improve infe

Question

TensorFlow website claims that Quantization provides up to 3x lower latency on mobile devices: https://www.tensorflow.org/lite/performance/post_training_quantization

I tried to verify this claim, and found that Quantized models are 45%-75% SLOWER than Float models in spite of being almost 4 times smaller in size. Needless to say, this is very disappointing and conflicts with Google's claims.

My test uses Google's official MnasNet model: https://storage.googleapis.com/mnasnet/checkpoints/mnasnet-a1.tar.gz

Here is the average latency based on 100 inference operations on a freshly rebooted phone:

• Pixel 2: float=81ms, quant=118ms
• Moto E: float=337ms, quant=590ms
• LG Treasure: float=547ms, quant=917ms

My test app measures the timing for only one method (tfLite.runForMultipleInputsOutputs). The results are very consistent (within 1% across multiple executions).

I am hoping to see some comments from the Tensorflow team or anybody who can share their metrics. The numbers above are based on image classifier model. I also tested an SSD MobileNetV2 object detector with similar results (quantized model being substantially slower).