Docker error : no space left on device

问题:

I installed docker on a Debian 7 machine in the following way

$ echo deb http://get.docker.io/ubuntu docker main > /etc/apt/sources.list.d/docker.list
$ sudo apt-get update
$ curl -sSL https://get.docker.com/ubuntu/ | sudo sh

After that when I first tried creating an Image it failed with the following error

 time="2015-06-02T14:26:37-04:00" level=info msg="[8] System error: write /sys/fs/cgroup/docker/01f5670fbee1f6687f58f3a943b1e1bdaec2630197fa4da1b19cc3db7e3d3883/cgroup.procs: no space left on device"

Here is the docker info

Containers: 2
Images: 21
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 25
Dirperm1 Supported: true
Execution Driver: native-0.2
Kernel Version: 3.16.0-0.bpo.4-amd64
Operating System: Debian GNU/Linux 7 (wheezy)
CPUs: 2
 Total Memory: 15.7 GiB


WARNING: No memory limit support
 WARNING: No swap limit support

How can I increase the memory? Where are the system configurations stored?

From Kal's suggestions:

When I got rid of all the images and containers it did free some space and the image build ran longer before failing with the same error. So the question is, which space is this referring to and how do I configure it?

回答1:

I had the same error and solve it this way:

1 . Delete the orphaned volumes in Docker, you can use the built-in docker volume command. The built-in command also deletes any directory in /var/lib/docker/volumes that is not a volume so make sure you didn't put anything in there you want to save.

Warning be very careful with this if you have some data you want to keep

Cleanup:

$ docker volume rm $(docker volume ls -qf dangling=true)

Additional commands:

List dangling volumes:

$ docker volume ls -qf dangling=true

List all volumes:

$ docker volume ls

---

2 . Also consider removing all the unused Images.

First get rid of the <none> images (those are sometimes generated while building an image and if for any reason the image building was interrupted, they stay there).

here's a nice script I use to remove them

docker rmi $(docker images | grep '^<none>' | awk '{print $3}')

Then if you are using Docker Compose to build Images locally for every project. You will end up with a lot of images usually named like your folder (example if your project folder named Hello, you will find images name Hello_blablabla). so also consider removing all these images

you can edit the above script to remove them or remove them manually with

docker rmi {image-name}

回答2:

UPDATE
The commands below have become hacks as Docker becomes more developed. The current best practice is

docker system prune

This will remove:

- all stopped containers
- all volumes not used by at least one container
- all networks not used by at least one container
- all dangling images

As below, this is nuclear.

---

To clean your system, first remove containers

$ docker rm $(docker ps -aq)

then remove images

$ docker rmi $(docker images -q)

This is of course nuclear and will remove all containers and all images. You can remove them one at at time via docker rm #CONTAINER_ID# and docker rmi #IMAGE_ID.

回答3:

Check that you have free space on /var as this is where Docker stores the image files by default (in /var/lib/docker).

First clean stuff up by using docker ps -a to list all containers (including stopped ones) and docker rm to remove them; then use docker images to list all the images you have stored and docker rmi to remove them.

Next change the storage location with a -g option on the docker daemon or by editing /etc/default/docker and adding the -g option to DOCKER_OPTS. -g specifies the location of the "Docker runtime" which is basically all the stuff that Docker creates as you build images and run containers. Choose a location with plenty of space as the disk space used will tend to grow over time. If you edit /etc/default/docker, you will need to restart the docker daemon for the change to take effect.

Now you should be able to create a new image (or pull one from Docker Hub) and you should see a bunch of files getting created in the directory you specified with the -g option.

回答4:

If it's just a test installation of Docker (ie not production) and you don't care about doing a nuclear clean, you can:

clean all containers: docker ps -a | sed '1 d' | awk '{print $1}' | xargs -L1 docker rm

clean all images: docker images -a | sed '1 d' | awk '{print $3}' | xargs -L1 docker rmi -f

Again, I use this in my ec2 instances when developing Docker, not in any serious QA or Production path. The great thing is that if you have your Dockerfile(s), it's easy to rebuild and or docker pull.

回答5:

Docker leaves dangling images around that can take up your space. To clean up after Docker, run the following:

docker image prune [-af if you want to force remove all images]

or with older versions of Docker:

docker rm $(docker ps -q -f 'status=exited')
docker rmi $(docker images -q -f "dangling=true")

This will remove exited and dangling images, which hopefully clears out device space.

回答6:

Your cgroups have the cpuset controller enabled. This controller is mostly useful in NUMA environment where it allows to finely specify which CPU/memory bank your tasks are allowed to run.

By default the mandatory cpuset.mems and cpuset.cpus are not set which means that there is "no space left" for your task, hence the error.

The easiest way to fix this is to enable cgroup.clone_children to 1 in the root cgroup. In your case, it should be

echo 1 > /sys/fs/cgroup/docker/cgroup.clone_children

It will basically instruct the system to automatically initialize container's cpuset.mems and cpuset.cpus from their parent cgroup.

回答7:

to remove all unused containers, volumes, networks and images at once (https://docs.docker.com/engine/reference/commandline/system_prune/#related-commands):

docker system prune -a -f

if it's not enough, one can remove running containers first:

docker rm -f $(docker ps -a -q)
docker system prune -a -f

increasing /var/lib/docker or using another location with more space is also a good alternative to get rid of this error (see How to change the docker image installation directory?)

回答8:

you can also use:

docker system prune

or for just volumes:

docker volume prune

回答9:

Clean Docker by using the following command:

docker images --no-trunc | grep '<none>' | awk '{ print $3 }' \
| xargs docker rmi

回答10:

1. Clean dangled images docker rmi $(docker images -f "dangling=true" -q)
2. Remove unwanted volumes
3. Remove unused images
4. Remove unused containers

回答11:

Seems like there are a few ways this can occur. The issue I had was that the docker disk image had hit its maximum size (Docker Whale -> Preferences -> Disk if you want to view what size that is in OSX).

I upped the limit and and was good to go. I'm sure cleaning up unused images would work as well.

回答12:

If you're using the boot2docker image via Docker Toolkit, then the problem stems from the fact that the boot2docker virtual machine has run out of space.

When you do a docker import or add a new image, the image gets copied into the /mnt/sda1 which might have become full.

One way to check what space you have available in the image, is to ssh into the vm and run df -h and check the remaining space in /mnt/sda1

The ssh command is docker-machine ssh default

Once you are sure that it is indeed a space issue, you can either clean up according to the instructions in some of the answers on this question, or you may choose to resize the boot2docker image itself, by increasing the space on /mnt/sda1

You can follow the instructions here to do the resizing of the image https://gist.github.com/joost/a7cfa7b741d9d39c1307

回答13:

In my case installation of ubuntu-server 18.04.1 [for some weird reason] created an LVM logical volume with just 4GBs in size instead of 750GBs. Therefore when pulling images i would get this "no space left on device" error. The fix is simple:

lvextend -l 100%FREE /dev/mapper/ubuntu--vg-ubuntu--lv
resize2fs /dev/mapper/ubuntu--vg-ubuntu--lv