Removing unused docker images on a remote Linux Az

Question

Introduction

Hi all, I set up a Docker machine in Azure using the following guides:

• https://blogs.msdn.microsoft.com/jcorioland/2016/08/19/build-push-and-run-docker-images-with-visual-studio-team-services/
• https://docs.docker.com/machine/drivers/azure/

I managed to get this all working and when doing a build on Windows I can now use the Linux Docker Host to create the docker image from the created sources. (By simply using the "Docker: Build an image" step followed by the "Docker: push an image" step with the DockerHostConnnection set to my Linux build machine:

This process however leaves the created images on disk.

What I wanted to do next is create a nightly build that cleans up the images. I basically created a new build and added the following steps:

1. Show docker images before: (docker images -a)
2. Docker Info: (docker info)
3. Remove docker images: (docker rmi $(docker images -aq))
4. Show docker images after: (docker images -a)

Problem

Whenever I run this build though I seem to receive the following error:

The strange thing is though, is that the command "docker images -aq" worked fine in the step before:

Investigation

I started doing some investigation by manually kicking off the commands from my own computer to the remote docker host but am running into the same problems with the following .cmd file:

docker.exe -H tcp://**********:2376 --tls --tlscacert='ca.pem' --tlscert='cert.pem' --tlskey='key.pem' login -u ********** -p ********** **********
docker.exe -H tcp://**********:2376 --tls --tlscacert='ca.pem' --tlscert='cert.pem' --tlskey='key.pem' images -aq
docker.exe -H tcp://**********:2376 --tls --tlscacert='ca.pem' --tlscert='cert.pem' --tlskey='key.pem' rmi $(docker images -a)
docker.exe -H tcp://**********:2376 --tls --tlscacert='ca.pem' --tlscert='cert.pem' --tlskey='key.pem' logout **********

Results of this .cmd file:

When running it locally from .cmd I see the same error:

Running it from PowerShell it works fine though:

Edit... (After more investigation)

Using this knowledge I modified the .cmd file to a .ps1 file which seemed to work a bit better. One problem was though that the $(docker ....) obtained it's information from the local docker installation. I changed the script to the following and now it works from my local machine to the Azure Linux docker host:

docker.exe -H tcp://**********:2376 --tls --tlscacert='ca.pem' --tlscert='cert.pem' --tlskey='key.pem' login -u ********** -p ********** **********
docker.exe -H tcp://**********:2376 --tls --tlscacert='ca.pem' --tlscert='cert.pem' --tlskey='key.pem' images -a
docker.exe -H tcp://**********:2376 --tls --tlscacert='ca.pem' --tlscert='cert.pem' --tlskey='key.pem' rmi -f $(docker -H tcp://**********:2376 --tls --tlscacert='ca.pem' --tlscert='cert.pem' --tlskey='key.pem' images -aq)
docker.exe -H tcp://**********:2376 --tls --tlscacert='ca.pem' --tlscert='cert.pem' --tlskey='key.pem' logout **********

So basically this now works, however, how can I get this to work with the Azure Docker build steps?

Because basically there seem to be 2 problems with the Azure Docker Integration steps:

1. I think the commands run in CMD instead of PowerShell which results in the -a error
2. When doing something like: 'docker -H .... rmi $(docker images -aq)', the second docker command (docker images...) will talk to the local docker instance. So I would actually like a solution for this without having to manually provide the IP address and all the certificates. (This problem I can probably solve in a hacky way though if I can't find a good solution)

Answer 1

It is running as command line and doesn’t support command nesting, you can use Shell Script task instead.

Similar thread: Command nesting in VSTS/TFS

Answer 2

I've modified the Docker VSTS task to support an output command so that chaining multiple Docker commands is possible.

I've created a Github repository with this and submitted a pull request: https://github.com/devedse/vsts-docker

In the following example I use the output variable to first obtain the Docker images and then remove them. Using the, just released, functionality of conditional tasks in VSTS I also managed to make VSTS build skip the "Remove the images" step whenever there is no image found.

Obtaining the Docker images:

Removing the Docker images if there are any found

Answer 3

This task does not allow to pipe output of one command to another. If we put input ps -aqf "status=exited" | ForEach-Object { docker rm -vf $_ } in the task input its actually running child_process.spawn(< path to docker exe>, ps -aqf "status=exited" | ForEach-Object { docker rm -vf $_ })

The whole input is passed as argument to docker.exe and hence redirecting output to another executable in same task is not possible. Even running another executable in task is not possible. This may be as design for security reason.

Same happens with simple cmd line. pipeing the output is not allowed and whole "-aqf "status=exited" | ForEach-Object { docker rm -vf $_ } " is passed as argument to docker.exe