How to include files outside of Docker's build

Question

How can I include files from outside of Docker's build context using the "ADD" command in the Docker file?

From the Docker documentation:

The path must be inside the context of the build; you cannot ADD ../something/something, because the first step of a docker build is to send the context directory (and subdirectories) to the docker daemon.

I do not want to restructure my whole project just to accommodate Docker in this matter. I want to keep all my Docker files in the same sub-directory.

Also, it appears Docker does not yet (and may not ever) support symlinks: Dockerfile ADD command does not follow symlinks on host #1676.

The only other thing I can think of is to include a pre-build step to copy the files into the Docker build context (and configure my version control to ignore those files). Is there a better workaround for than that?

Answer 1

I spent a good time trying to figure out a good pattern and how to better explain what's going on with this feature support. I realized that the best way to explain it was as follows...

• Dockerfile: Will only see files under its own relative path
• Context: a place in "space" where the files you want to share and your Dockerfile will be copied to

So, with that said, here's an example of the Dockerfile that needs to reuse a file called start.sh

Dockerfile

It ALWAYS will load from its relative path, having the current dir of itself as the local reference to the paths you specify.

COPY start.sh /runtime/start.sh

Files

Considering this idea, we can think of having multiple copies for the Dockerfiles building specific things, but they all need access to the start.sh.

./all-services/
   /start.sh
   /service-X/Dockerfile
   /service-Y/Dockerfile
   /service-Z/Dockerfile
./docker-compose.yaml

Considering this structure and the files above, here's a docker-compose.yml

docker-compose.yaml

• In this example, your shared context dir is the runtime dir.
  • Same mental model here, think that all the files under this dir are moved over to the so-called context.
  • Similarly, just specify the Dockerfile that you want to copy to that same dir. You can specify that using dockerfile.
• the directory where your main content is located is the actual context to be set.

The docker-compose.yml is as follows

version: "3.3"
services:

  service-A
    build:
      context: ./all-service
      dockerfile: ./service-A/Dockerfile

  service-B
    build:
      context: ./all-service
      dockerfile: ./service-B/Dockerfile

  service-C
    build:
      context: ./all-service
      dockerfile: ./service-C/Dockerfile

• all-service is set as the context, the shared file start.sh is copied there as well the Dockerfile specified by each dockerfile.
• Each gets to be built their own way, sharing the start file!

Cheers!

Answer 2

I often find myself utilizing the --build-arg option for this purpose. For example after putting the following in the Dockerfile:

ARG SSH_KEY
RUN echo "$SSH_KEY" > /root/.ssh/id_rsa

You can just do:

docker build -t some-app --build-arg SSH_KEY="$(cat ~/file/outside/build/context/id_rsa)" .

Answer 3

The best way to work around this is to specify the Dockerfile independently of the build context, using -f.

For instance, this command will give the ADD command access to anything in your current directory.

docker build -f docker-files/Dockerfile .

Update: Docker now allows having the Dockerfile outside the build context (fixed in 18.03.0-ce, https://github.com/docker/cli/pull/886). So you can also do something like

docker build -f ../Dockerfile .

Answer 4

If you read the discussion in the issue 2745 not only docker may never support symlinks they may never support adding files outside your context. Seems to be a design philosophy that files that go into docker build should explicitly be part of its context or be from a URL where it is presumably deployed too with a fixed version so that the build is repeatable with well known URLs or files shipped with the docker container.

I prefer to build from a version controlled source - ie docker build -t stuff http://my.git.org/repo - otherwise I'm building from some random place with random files.

fundamentally, no.... -- SvenDowideit, Docker Inc

Just my opinion but I think you should restructure to separate out the code and docker repositories. That way the containers can be generic and pull in any version of the code at run time rather than build time.

Alternatively, use docker as your fundamental code deployment artifact and then you put the dockerfile in the root of the code repository. if you go this route probably makes sense to have a parent docker container for more general system level details and a child container for setup specific to your code.

Answer 5

You can also create a tarball of what the image needs first and use that as your context.

https://docs.docker.com/engine/reference/commandline/build/#/tarball-contexts

Answer 6

On Linux you can mount other directories instead of symlinking them

mount --bind olddir newdir

See https://superuser.com/questions/842642 for more details.

I don't know if something similar is available for other OSes. I also tried using Samba to share a folder and remount it into the Docker context which worked as well.