How do you add items to .dockerignore?

Question

I'm not able to find many examples of what a .dockerignore file should look like.

Using puppet to install a few packages on a docker container causes the image to explode from 600MB to 3GB. I'm trying to use a .dockerignore file to keep the size to a minumum

$ cat Dockerfile  
FROM centos:centos6

#Work around selinux problem on cent images
RUN yum install -y --enablerepo=centosplus libselinux-devel

RUN yum install -y wget git tar openssh-server; yum -y clean all

Add Puppetfile / 
RUN librarian-puppet install
RUN puppet apply --modulepath=/modules -e "class { 'buildslave': jenkins_slave => true,}"
RUN librarian-puppet clean

If I run docker images --tree I can see that the image instantlly grows by several GB

$ docker images --tree
                ├─e289570b5555 Virtual Size: 387.7 MB
                │ └─a7646acf90d0 Virtual Size: 442.5 MB
                │   └─d7bc6e1fbe43 Virtual Size: 442.5 MB
                │     └─772e6b204e3b Virtual Size: 627.5 MB
                │       └─599a7b5226f4 Virtual Size: 627.5 MB
                │         └─9fbffccda8bd Virtual Size: 2.943 GB
                │           └─ee46af013f6b Virtual Size: 2.943 GB
                │             └─3e4fe065fd07 Virtual Size: 2.943 GB
                │               └─de9ec3eba39e Virtual Size: 2.943 GB
                │                 └─31cba2716a12 Virtual Size: 2.943 GB
                │                   └─52cbc742d3c4 Virtual Size: 2.943 GB
                │                     └─9a857380258c Virtual Size: 2.943 GB
                │                       └─c6d87a343807 Virtual Size: 2.964 GB
                │                         └─f664124e0080 Virtual Size: 2.964 GB
                │                           └─e6cc212038b9 Virtual Size: 2.964 GB Tags: foo/jenkins-centos6-buildslave:latest

I believe the reason that the image grows so large, is because librarian-puppet clones a puppet module to /modules which breaks the build cache

I've tried the following .dockerignore files with no luck.

$ cat .dockerignore
/modules
/modules/
/modules/*

Is this the correct syntax for a .dockerignore file?
Are there any other ways to prevent these containers from growing so large?

Additional information:

http://kartar.net/2013/12/building-puppet-apps-inside-docker/
http://danielmartins.ninja/posts/a-week-of-docker.html

Answer 1

The format of the .dockerignore should be equal to the one of .gitignore. See a sample file and the docker documentation.

The file should be a list of exclusion patterns (relative to the path of the .dockerignore file) separated by a newline.

So you should try the following .dockerignore:

modules/*

The / at the beginning may have been the mistake, as it will only be valid for the root directory of the file (but not for subdirectories, so maybe the recursive version without the / will do a better job instead).

Answer 2

http://docs.docker.com/articles/dockerfile_best-practices/

It seems to me your approach is backwards (agreeing with @csanchez), and that you should be generating your docker container from puppet, not running puppet in the container...

Also, you should && the install/apply/clean lines together... each docker command creates an incremental image... If there are temporary/resource files that are part of the centos yum commands, you should likewise do the same.

FROM centos:centos6

# Add your needed files first
# maybe you could use a baseimage and make this a volume mount point?
Add Puppetfile / 

# combine multiple commands with cleanup of cache/temporary
# space in the same run sequence to reduce wasted diff image space.

#Work around selinux problem on cent images
RUN yum install -y --enablerepo=centosplus libselinux-devel && \
    yum install -y wget git tar openssh-server; yum -y clean all && \
    librarian-puppet install && \
    puppet apply --modulepath=/modules -e "class { 'buildslave': jenkins_slave => true,}" && \
    librarian-puppet clean

I'd REALLY suggest avoiding SELINUX in a container, it doesn't give you anything inside a container. Not to mention, that depending on what you are trying to create, there are smaller places to start from than centos6. I believe ubuntu is smaller, debian:wheezy smaller still, or even alpine for tiny start point.

It is worth noting, that your file size, if you're using a file system that supports virtual mounts, can reuse the same base image for multiple instances, so it won't grown more

Answer 3

The .dockerignore file is similar to the .gitignore syntax. Here are some example rules:

# Ignore a file or directory in the context root named "modules"
modules

# Ignore any files or directories within the subdirectory named "modules" 
# in the context root
modules/*

# Ignore any files or directories in the context root beginning with "modules"
modules*

# Ignore any files or directories one level down from the context root named
# "modules"
*/modules

# Ignore any files or directories at any level, including the context root, 
# named modules
**/modules

# Ignore every file in the entire build context (see next rule for how this 
# could be used)
*

# Re-include the file or directory named "src" that may have been previously
# excluded. Note that you cannot re-include files in subdirectories that have 
# been previously excluded at a higher level
!src

Note that "build context" is the directory you pass at the end of your build command, typically a . to indicate the current directory. This directory is packaged from the docker client, excluding any files you have ignored with .dockerignore, and sent to the docker daemon to perform the build. Even when the daemon is on the same host as your client, the build only works from this context and not directly from the folders.

There is only a single .dockerignore for a build, and it must be in the root of the build context. It will not work if it is in your home directory (assuming you build from a subdirectory), and it will not work from a subdirectory of your build context.

To test what is in your current build context and verify your .dockerignore file is behaving correctly, you can copy/paste the following (this assumes you do not have an image named test-context, it will be overwritten and then deleted if you do):

# create an image that includes the entire build context
docker build -t test-context -f - . <<EOF
FROM busybox
COPY . /context
WORKDIR /context
CMD find .
EOF

# run the image which executes the find command
docker container run --rm test-context

# cleanup the built image
docker image rm test-context

Answer 4

.dockerignore is to prevent files from being added to the initial build context that is sent to the docker daemon when you do docker build, it doesn't create a global rule for excluding files from being created in all images generated by a Dockerfile.

It's important to note that each RUN statement will generate a new image, with the parent of that image being the image generated by the Dockerfile statement above it. Try collapsing your RUN statements into a single one to reduce image size:

RUN librarian-puppet install &&\
 puppet apply --modulepath=/modules -e "class { 'buildslave': jenkins_slave => true,}" &&\
 librarian-puppet clean

Answer 5

Optimizing container image size is the main goal behind the .dockerignore as it serve a purpose similar to your .gitignore as it reduces the latency and response time while providing services. It is true for deployment automation such as Puppet, SaltStack or Ansible. Timestamp defined for service execution deployment may be failed because of larger image size and low network bandwidth. So .dockerignore helps to make the size of image as small as possible.

You could place it into the build context directory which we specify at the end of a docker build command. The file follows glob pattern for files and directories to exclude those from the final build image.

Suppose I have a directory .img/ into my build context, and I want to exclude it while building image, I'll simply add the following line into .dockerignore file,

.img

And, if I want to exclude all files starts with . then simply, add the line,

.*

(Note: Don't confuse the Unix glob pattern is different than Regular expressions)

In addition, I'll exclude few more of my files from my build context,

.*
docs
my-stack.dab
docker-compose.overrride.yml
test*
*.md
!README.md

Here, *.md line excludes all markdown files(I have many markdown files into my project). But, I want to include README.md and no other markdown files. As our last line in above, we have added README.md with ! or exclude it while excluding all other markdown files.

So, with this we can reduce the overhead of your build image with the help of .dockerignore and leverage to make image size smaller.

Answer 6

A different way of doing it, creating a smaller image, is to run librarian-puppet in the host, not in Docker, so you don't end with librarian, ruby, gems,... installed in the image.

I ended with a 622MB image for jenkins slave using Puppet, and a 480MB image without Puppet.