How should I backup & restore docker named volumes

Question

I'm a little bit confused with the functionnality of named volumes in a docker compose file specifically when it comes to backup/restore my app.

I'm actually testing this dockercompose file :

      version: '2'
      services:
          django:
              build: 
                  context: "{{ build_dir }}/docker/django"
              depends_on:
                  - db
              environment:
                  [...]
              volumes:
                  - code:/data/code
                  - www:/var/www
                  - conf:/data/conf
              networks:
                  - front
                  - db
              expose:
                  - "8080"
              entrypoint: "/init"
          db:
              build:
                  context: "{{ build_dir }}/docker/postgres" 
              environment:
                  [...]
              volumes:
                  - data:/var/lib/postgresql/data
              networks:
                  - db

      volumes:
          data:
          www:
          code:
          conf:

      networks:
          front:
              external:
                  name: "proxy_nw"

As the documentation said, I tried to use named volume instead of data only container. But how am I suppose to backup my data ?

With a data only container I would have done a docker run --rm --volume-from DOC backup_container save which is really easy.

Now I read in this topic that I should use something like docker run --rm --volume data --volume www --volume code --volume conf backup_container save. This is not so simple because I have many applications with different types and names of volumes so it means that my command to save my data would have to be different for each application. It complicate automation process.

Edit: Actually this syntaxe docker run --volume data --volume www container_image my_command is not correct. It needs the mountpoint inside the container, so it would be docker run --volume data:/somewhere --volume www:/somewhereelse container_image my_command. So it's even more complicated to use with a backup container.

So, what are the best practices in this case ? Should I use just one named volume for all my containers ?

Answer 1

Here is an example of how I backup and restore mongo volume data as docker image.

# let a9c25f42ccca be mongo container id    
docker stop a9c25f42ccca

# create a temp container with volume taken from mongo 
# make a local tar archive inside it
docker run --name temp_backup --volumes-from a9c25f42ccca ubuntu tar cvf /mongo_backup.tar /data/db

docker start a9c25f42ccca

# make an image and remove temp container
docker commit temp_backup my_mongo_backup
docker rm temp_backup

# push image with backup to remote registry if needed
docker push my_mongo_backup

And restoring.

#pull image if needed
docker pull my_mongo_backup

docker stop a9c25f42ccca

# run transient container out from image with backup
# take volume from mongo
# clear any existing data
# restore data from tar arhcive
docker run --rm --volumes-from a9c25f42ccca my_mongo_backup bash -c "rm -rf /data/db/* && tar xvf /mongo_backup.tar -C /data --strip 1"

docker start a9c25f42ccca

Answer 2

Actually it should be done same way as written in official documentation. Data volume container stores it's data in "virtual root", so you should backup with next command:

docker run --rm \ 
  --volume [DOCKER_COMPOSE_PREFIX]_[VOLUME_NAME]:/[TEMPORARY_DIRECTORY_TO_STORE_VOLUME_DATA] \
  --volume $(pwd):/[TEMPORARY_DIRECTORY_TO_STORE_BACKUP_FILE] \
  ubuntu \
  tar cvf /[TEMPORARY_DIRECTORY_TO_STORE_BACKUP_FILE]/[BACKUP_FILENAME].tar /[TEMPORARY_DIRECTORY_TO_STORE_VOLUME_DATA]

where:

• --rm means that the image created for this run command will be cleaned up
• DOCKER_COMPOSE_PREFIX in default is your project directory name
• VOLUME_NAME is the data-volume container name from compose file
• TEMPORARY_DIRECTORY_TO_STORE_VOLUME_DATA is a directory to mount your volume data
• TEMPORARY_DIRECTORY_TO_STORE_BACKUP_FILE is a directory virtually mapped to your current directory, where the backup will be placed
• BACKUP_FILENAME - a name of backup file (you find it in current directory)
• ubuntu - you may change image type to another container with tar :)

Get data back into the volume(restore):

docker run --rm \ 
  --volume [DOCKER_COMPOSE_PREFIX]_[VOLUME_NAME]:/[TEMPORARY_DIRECTORY_STORING_EXTRACTED_BACKUP] \
  --volume $(pwd):/[TEMPORARY_DIRECTORY_TO_STORE_BACKUP_FILE] \
  ubuntu \
  tar xvf /[TEMPORARY_DIRECTORY_TO_STORE_BACKUP_FILE]/[BACKUP_FILENAME].tar -C /[TEMPORARY_DIRECTORY_STORING_EXTRACTED_BACKUP] --strip 1

where:

• TEMPORARY_DIRECTORY_STORING_EXTRACTED_BACKUP is a directory where the extracted files will be copied to (this is linked with the volume and will therefore write to it)
• -C - tell tar where to extract the contents
• --strip 1 - remove leading path elements (e.g. the parent directory if the backup contents are located in a /temp folder or similar)

Answer 3

I finally changed my approach. I parse the volumes of my containers looking for the mountpoints and I backup everything in a separate folder in the tar.

So I'm not using a container to do it but an external script. I don't know if it's a better approach but it works well.