How to automatically update your docker containers

Question

Say I have a trivial container based on the ubuntu:latest. Now there is a security update and ubuntu:latest is updated in the docker repo .

1. How would I know my local image and its containers are running behind?

2. Is there some best practice for automatically updating local images and containers to follow the docker repo updates, which in practice would give you the same niceties of having unattended-upgrades running on a conventional ubuntu-machine

Answer 1

You would not know your container is behind without running docker pull. Then you'd need to rebuild or recompose your image.

docker pull image:tag
docker-compose -f docker-compose.yml -f production.yml up -d --build

The commands can be put in a script along with anything else necessary to complete the upgrade, although a proper container would not need anything additional.

Answer 2

One of the ways to do it is to drive this through your CI/CD systems. Once your parent image is built, have something that scans your git repos for images using that parent. If found, you'd then send a pull request to bump to new versions of the image. The pull request, if all tests pass, would be merged and you'd have a new child image based on updated parent. An example of a tool that takes this approach can be found here: https://engineering.salesforce.com/open-sourcing-dockerfile-image-update-6400121c1a75 .

If you don't control your parent image, as would be the case if you are depending on the official ubuntu image, you can write some tooling that detects changes in the parent image tag and invoke children image builds accordingly.

Answer 3

There are a lot of answers here, but none of them suited my needs. I wanted an actual answer to the asker's #1 question. How do I know when an image is updated on hub.docker.com?

The below script can be run daily. On first run, it gets a baseline of the tags and update dates from the HUB registry and saves them locally. From then out, every time it is run it checks the registry for new tags and update dates. Since this changes every time a new image exists, it tells us if the base image has changed. Here is the script:

#!/bin/bash

DATAPATH='/data/docker/updater/data'

if [ ! -d "${DATAPATH}" ]; then
        mkdir "${DATAPATH}";
fi
IMAGES=$(docker ps --format "{{.Image}}")
for IMAGE in $IMAGES; do
        ORIGIMAGE=${IMAGE}
        if [[ "$IMAGE" != *\/* ]]; then
                IMAGE=library/${IMAGE}
        fi
        IMAGE=${IMAGE%%:*}
        echo "Checking ${IMAGE}"
        PARSED=${IMAGE//\//.}
        if [ ! -f "${DATAPATH}/${PARSED}" ]; then
                # File doesn't exist yet, make baseline
                echo "Setting baseline for ${IMAGE}"
                curl -s "https://registry.hub.docker.com/v2/repositories/${IMAGE}/tags/" > "${DATAPATH}/${PARSED}"
        else
                # File does exist, do a compare
                NEW=$(curl -s "https://registry.hub.docker.com/v2/repositories/${IMAGE}/tags/")
                OLD=$(cat "${DATAPATH}/${PARSED}")
                if [[ "${VAR1}" == "${VAR2}" ]]; then
                        echo "Image ${IMAGE} is up to date";
                else
                        echo ${NEW} > "${DATAPATH}/${PARSED}"
                        echo "Image ${IMAGE} needs to be updated";
                        H=`hostname`
                        ssh -i /data/keys/<KEYFILE> <USER>@<REMOTEHOST>.com "{ echo \"MAIL FROM: root@${H}\"; echo \"RCPT TO: <USER>@<EMAILHOST>.com\"; echo \"DATA\"; echo \"Subject: ${H} - ${IMAGE} needs update\"; echo \"\"; echo -e \"\n${IMAGE} needs update.\n\ndocker pull ${ORIGIMAGE}\"; echo \"\"; echo \".\"; echo \"quit\"; sleep 1; } | telnet <SMTPHOST> 25"
                fi

        fi
done;

You will want to alter the DATAPATH variable at the top, and alter the email notification command at the end to suit your needs. For me, I have it SSH into a server on another network where my SMTP is located. But you could easily use the mail command, too.

Now, you also want to check for updated packages inside the containers themselves. This is actually probably more effective than doing a "pull" once your containers are working. Here's the script to pull that off:

#!/bin/bash


function needsUpdates() {
        RESULT=$(docker exec ${1} bash -c ' \
                if [[ -f /etc/apt/sources.list ]]; then \
                grep security /etc/apt/sources.list > /tmp/security.list; \
                apt-get update > /dev/null; \
                apt-get upgrade -oDir::Etc::Sourcelist=/tmp/security.list -s; \
                fi; \
                ')
        RESULT=$(echo $RESULT)
        GOODRESULT="Reading package lists... Building dependency tree... Reading state information... Calculating upgrade... 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded."
        if [[ "${RESULT}" != "" ]] && [[ "${RESULT}" != "${GOODRESULT}" ]]; then
                return 0
        else
                return 1
        fi
}

function sendEmail() {
        echo "Container ${1} needs security updates";
        H=`hostname`
        ssh -i /data/keys/<KEYFILE> <USRER>@<REMOTEHOST>.com "{ echo \"MAIL FROM: root@${H}\"; echo \"RCPT TO: <USER>@<EMAILHOST>.com\"; echo \"DATA\"; echo \"Subject: ${H} - ${1} container needs security update\"; echo \"\"; echo -e \"\n${1} container needs update.\n\n\"; echo -e \"docker exec ${1} bash -c 'grep security /etc/apt/sources.list > /tmp/security.list; apt-get update > /dev/null; apt-get upgrade -oDir::Etc::Sourcelist=/tmp/security.list -s'\n\n\"; echo \"Remove the -s to run the update\"; echo \"\"; echo \".\"; echo \"quit\"; sleep 1; } | telnet <SMTPHOST> 25"
}

CONTAINERS=$(docker ps --format "{{.Names}}")
for CONTAINER in $CONTAINERS; do
        echo "Checking ${CONTAINER}"
        if needsUpdates $CONTAINER; then
                sendEmail $CONTAINER
        fi
done

Answer 4

We use a script which checks if a running container is started with the latest image. We also use upstart init scripts for starting the docker image.

#!/usr/bin/env bash
set -e
BASE_IMAGE="registry"
REGISTRY="registry.hub.docker.com"
IMAGE="$REGISTRY/$BASE_IMAGE"
CID=$(docker ps | grep $IMAGE | awk '{print $1}')
docker pull $IMAGE

for im in $CID
do
    LATEST=`docker inspect --format "{{.Id}}" $IMAGE`
    RUNNING=`docker inspect --format "{{.Image}}" $im`
    NAME=`docker inspect --format '{{.Name}}' $im | sed "s/\///g"`
    echo "Latest:" $LATEST
    echo "Running:" $RUNNING
    if [ "$RUNNING" != "$LATEST" ];then
        echo "upgrading $NAME"
        stop docker-$NAME
        docker rm -f $NAME
        start docker-$NAME
    else
        echo "$NAME up to date"
    fi
done

And init looks like

docker run -t -i --name $NAME $im /bin/bash

Answer 5

A simple and great solution is shepherd

Answer 6

Dependency management for Docker images is a real problem. I'm part of a team that built a tool, MicroBadger, to help with this by monitoring container images and inspecting metadata. One of its features is to let you set up a notification webhook that gets called when an image you're interested in (e.g. a base image) changes.