Docker how to ADD a file without a commiting it to

Question

I have a ~300Mb zipped local file that I add to a docker image. The next state then extracts the image.

The problem is that the ADD statement results in a commit that results in a new file system layer makes the image ~300Mb larger than it needs to be.

ADD /files/apache-stratos.zip /opt/apache-stratos.zip
RUN unzip -q apache-stratos.zip && \
    rm apache-stratos.zip && \
    mv apache-stratos-* apache-stratos

Question: Is there a work-around to ADD local files without causing a commit?

One option is to run a simple web server (e.g. python -m SimpleHTTPServer) before starting the docker build, and then using wget to retrieve the file, but that seems a bit messy:

RUN wget http://localhost:8000/apache-stratos.zip && \
    unzip -q apache-stratos.zip && \
    rm apache-stratos.zip && \
    mv apache-stratos-* apache-stratos

Another option is to extract the zipped file at container start up instead of build time, but I would prefer to keep the start up as quick as possible.

Answer 1

With Docker 17.05+ you can use a multi-stage build to avoid creating extra layers.

FROM ... as stage1
# No need to clean up here, these layers will be discarded
ADD /files/apache-stratos.zip /opt/apache-stratos.zip
RUN unzip -q apache-stratos.zip
    && mv apache-stratos-* apache-stratos

FROM ...
COPY --from=stage1 apache-stratos/ apache-stratos/

Answer 2

According to the documentation, if you pass an archive file from the local filesystem (not a URL) to ADD in the Dockerfile (with a destination path, not a path + filename), it will uncompress the file into the directory given.

If <src> is a local tar archive in a recognized compression format (identity, gzip, bzip2 or xz) then it is unpacked as a directory. Resources from remote URLs are not decompressed. When a directory is copied or unpacked, it has the same behavior as tar -x: the result is the union of:

1) Whatever existed at the destination path and 2) The contents of the source tree, with conflicts resolved in favor of "2." on a file-by-file basis.

try:

ADD /files/apache-stratos.zip /opt/

and see if the files are there, without further decompression.

Answer 3

You can use docker-squash to squash newly created layers. That should reduce the image size significantly.

Unfortunately the mentioned workarounds (RUN curl ... && unzip ... & rm ..., unpack on container start) are the only options at the moment (docker 1.11).