gcloud docker push reliability

Question

I have been having a lot of problems pushing images with gcloud docker push over the past few weeks. I've read through the many stack overflow discussions and github issues and workarounds but I haven't come across a solution to the inconsistency yet.

Typically I will attempt to push a container image or two. The first push will almost always fail with the following retry-until-timeout output:

I can only get around it with gcloud auth login. At most 5 minutes later I will attempt to push a second image, and will again see the retry-until-timeout issue. I will see this on every attempt until I gcloud auth login again.

Often I will have to manually retry several more times immediately after authenticating before the image is actually pushed.

1. Am I actually being logged out (I can still access pods and instances, etc with kubectl and gcloud machines)? If so, why is being logged out inconsistent and what does building docker containers do that it would invalidate my local gcloud session?
2. If not, why can't I gcloud docker push until I authenticate again? After that, why is this still inconsistent (I suspect it may have little or nothing to do with the real issue).
3. Is there a way to make pushing images on OSX with docker-machine and gcloud docker push reliable? Is there another way to get images to the cloud repository (preferably from the command line)?

gcloud --version alpha 2016.01.12 beta 2016.01.12 bq 2.0.18 bq-nix 2.0.18 core 2016.02.11 core-nix 2016.02.05 gcloud gsutil 4.16 gsutil-nix 4.15 kubectl kubectl-darwin-x86_64 1.1.7

docker --version Docker version 1.10.1, build 9e83765

docker-machine --version docker-machine version 0.6.0, build e27fb87

virtualbox version 5.0.14 r105127

Answer 1

I had the same or similar problem. After a few minutes of retry loop depicted with screenshoot above, the command will fail with net/http: TLS handshake timeout.

The solution that fixed it for me was editing the docker daemon configuration with

DOCKER_OPTS="--max-concurrent-uploads=1"

I had a feeling this issue was connected with docker clogging up the network, as I noticed even browsing to gmail can get a timeout(!)

Answer 2

Switching to regular docker push doesn't help timeouts. This appears to be related to your ISP and uploading assets.

I was receiving the same error. After moving the Docker build process to the cloud (which has a much larger pipeline), gcloud docker builds and deploys the image just fine.

Answer 3

I never faced the problems you mentioned with gcloud docker, but regarding your last point,

Is there another way to get images to the cloud repository (preferably from the command line)?

it is indeed possible to push to the gcr.io repos without going through gcloud, e.g:

docker login -e [e-mail] -p $(gcloud auth print-access-token) -u _token https://gcr.io
docker push [your-image]

Credits to mattmoor, more info in original answer here: Access google container registry without the gcloud client