Docker daemon flags ignored

Question

Environment:

• OS: debian 8.0.0-amd64, ubuntu-15.04, 16.04
• Docker: 1.x.x

Procedure:

I changed /etc/default/docker to add a private docker registry, then I restarted docker service and finally tried to pull some image.

$ cat /etc/default/docker
DOCKER_OPTS="--insecure-registry mydocker-registry.net:5000"

$ service docker restart

$ docker pull mydocker-registry.net:5000/testdb
FATA[0000] Error: v1 ping attempt failed with error: Get https://mydocker-
registry.net:5000/v1/_ping: dial tcp: lookup mydocker-registry.net: no 
such host. If this private registry supports only HTTP or HTTPS with an 
unknown CA certificate, please add `--insecure-registry mydocker-
registry.net:5000` to the daemon's arguments. In the case of HTTPS, if 
you have access to the registry's CA certificate, no need for the flag; 
simply place the CA certificate at /etc/docker/certs.d/mydocker-
registry.net:5000/ca.crt

A ps output shows nothing about DOCKER_OPTS environment var.

$ ps auxwww|grep docker
root  6919   0.0   0.1   331076   19984 ? Ssl 10:14   0:00 /usr/bin/docker -d -H fd://

Question:

According to docker documentation the way to use a private registry is through DOCKER_OPTS in /etc/default/docker. Why, after doing that, it does not take effect in this environment?

Notes:

• The private registry hostname is correctly resolved by the DNS.

Answer 1

Systemd is really not designed for appending options to ExecStart or Environment. The best and also most platform-independent way is to use the /etc/docker/daemon.json configuration file.

Behold:

cat > /etc/docker/daemon.json <<DOCKERCONFIG
{
  "labels": ["foo=bar"],
  "insecure-registries": ["mydocker-registry.net:5000"]
}
DOCKERCONFIG

Answer 2

Things seem to have changed in Ubuntu 16.04 using docker 1.12.x. Based on the updated documentation

Add DOCKER_OPTS="-g /mnt/somewhere/else/docker/ --storage-driver=overlay2" to /etc/default/docker

Edit file /lib/systemd/system/docker.service

...
[Service]
ExecStart=/usr/bin/dockerd -H fd:// $DOCKER_OPTS
...
EnvironmentFile=-/etc/default/docker
...

Then execute:

sudo systemctl daemon-reload
sudo systemctl restart docker

Answer 3

Recommended Way Docker 17.xx +

There are a number of ways to configure the daemon flags and environment variables for your Docker daemon. The recommended way is to use the platform-independent daemon.json file, which is located in /etc/docker/ on Linux by default.

So, for configuring insecure registries, do the following:

1. Set the following flag in the /etc/docker/daemon.json file:

   {
       "insecure-registries": ["mydocker-registry.net:5000"]
   }
   

2. Restart Docker

    $ sudo systemctl restart docker
   

Easier each time!

---

Previously Recommended Way with Docker 1.12

According to docker documentation, The recommended way to configure the daemon flags and environment variables for your Docker daemon is to use a systemd drop-in file.

So, for this specific case, do the following:

1. Create a file called /etc/systemd/system/docker.service.d/private-registry.conf with the following content:

   If not exists, create directory /etc/systemd/system/docker.service.d

   [Service]
   ExecStart=
   ExecStart=/usr/bin/dockerd --insecure-registry mydocker-registry.net:5000
   

2. Flush changes:

   $ sudo systemctl daemon-reload
   

3. Restart Docker:

    $ sudo systemctl restart docker
   

Voila!

---

Not recommended way

Edit file /lib/systemd/system/docker.service

...
[Service]
ExecStart=/usr/bin/docker -d -H fd:// $DOCKER_OPTS
...
EnvironmentFile=-/etc/default/docker
...

Then execute

systemctl daemon-reload
systemctl restart docker

Verify that /etc/default/docker is loaded

ps auxwww | grep docker
root      4989  0.8  0.1 265540 16608 ?        Ssl  10:37   0:00 /usr/bin/docker -d -H fd:// --insecure-registry 

That's it.

Answer 4

Ubuntu specific solution to insecure-registry via DOCKER_OPTS

Because...

$ dpkg --list | grep -i docker
ii  docker.io                          1.12.3-0ubuntu4~16.04.2            amd64        Linux container runtime

...ships with...

$ cat /etc/systemd/system/multi-user.target.wants/docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target docker.socket
Requires=docker.socket

[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
EnvironmentFile=-/etc/default/docker
ExecStart=/usr/bin/dockerd -H fd:// $DOCKER_OPTS
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process

[Install]
WantedBy=multi-user.target

...(Specifically: ExecStart=/usr/bin/dockerd -H fd:// $DOCKER_OPTS) you can do a hybrid approach combining the [chosen answer's] "Recommended Way" and the use of DOCKER_OPTS to keep from blowing over the -H fd:// option if you were to redefine ExecStart

# The docker.io package doesn't create a systemd drop-ins directory, so we will
$ mkdir -p /etc/systemd/system/docker.service.d
$ cat > /etc/systemd/system/docker.service.d/10-insecure-registry.conf <<EOF
[Service]
Environment="DOCKER_OPTS=--insecure-registry docker.internal:5000"
EOF

Answer 5

I had a similar challenge. When I started looking to begin moving some systems from Ubuntu 14.04 to Ubuntu 16.04. My goal was to use one dockerd configuration file with dockerd flags (DOCKER_OPTS) for both Ubuntu 16.04 (systemd) and Ubuntu 14.04 (Upstart) other than /etc/docker/daemon.json. I chose not to use /etc/docker/daemon.json for docker daemon configuration because json does not support comments.

I wanted a systemd design to use an override file, which only modifies dockerd flags. It uses the default Docker systemd configuration file (/lib/systemd/system/docker.service) for other Docker settings. Another objective was to customise systemd on each system after each change or boot.

It solves my challenge. It may help you.

https://github.com/BradleyA/docker-scripts/tree/master/dockerd-configuration-options

git clone https://github.com/BradleyA/docker-scripts
cd docker-scripts/dockerd-configuration-options

Answer 6

Systemd based systems do not read /etc/default configurations, you have to put those in /etc/systemd now, see also docker bug docker bug #12926

There is an official documentation on the Docker site now, refer to Control and configure Docker with systemd.

You should never directly hack the service files for configuration.

Tested and works on Arch and Debian based systems - I had to include the option to ignore any obsolete EnvironmentFile directives, though (see also linked Docker reference, but I didn't spot it at first and thought it was not needed):

-EnvironmentFile=/etc/default/docker
ExecStart=
ExecStart=/usr/bin/docker daemon ...