I am using Google Cloud Vision API on my Raspberry PI. It works fine when I use it on my home (on which the cloud account was first accessed) network but if I access the API from a different network it raises a token refresh error. I have synchronized the time using NTP but is of no help.
Detailed error:
oauth2client.client.HttpAccessTokenRefreshError: invalid_grant: Invalid JWT: Token must be a short-lived token (60 minutes) and in a reasonable timeframe. Check your iat and exp values and use a clock with skew to account for clock differences between systems.
This is caused by poor synchronisation of the computer's clock where the code was executed that had a lag of 5 minutes or more. The error means that the token is not valid and its not valid because the time on the PI is different then the actual time. I know that you say you are checking NTP but I suggest you check timezone as well.
I know this is a PI so debugging is probably limited but try outputting your id token and decryption its a jwt you should be able to see the iat and exp values. It might help you figure out what time it thinks it is.