How to prevent Dockerfile caching git clone

Question

I have a Dockerfile trying to package and deploy a web app to a container. The code of app fetches from git repository during Docker image building. Here's the Dockerfile snapshot:

........
RUN git clone --depth=1 git-repository-url $GIT_HOME/
RUN mvn package -Dmaven.test.skip
........

I want the docker do not cache the step of RUN git clone --depth=1 git-repository-url $GIT_HOME/ so that the on-going updated on the the repository can be reflected on the Docker image building. Is it possible to a achieve that?

Answer 1

If you use github you can use github API to not cache specific RUN command. You need to have jq installed to parse JSON: apt-get install -y jq

Example:

docker build --build-arg SHA=$(curl -s 'https://api.github.com/repos/Tencent/mars/commits' | jq -r '.[0].sha') -t imageName .

In Dockerfile (ARG command should be right before RUN):

ARG SHA=LATEST
RUN SHA=${SHA} \
    git clone https://github.com/Tencent/mars.git

or if you don't want to install jq

SHA=$(curl -s 'https://api.github.com/repos/Tencent/mars/commits' | grep sha | head -1)

If repository has new commits, git clone will be executed.

Answer 2

I ran into this same issue myself, and I just decided to use the --no-cache option when I build the image, rather than trying to single out the git repo.

docker build --no-cache -t my_image .

Answer 3

Another workaround:

If you use github (or gitlab or bitbucket too most likely) you can ADD the github API's representation of your repo to a dummy location.

   ADD https://api.github.com/repos/$USER/$REPO/git/refs/heads/$BRANCH version.json
   RUN git clone -b$BRANCH https://github.com/$USER/$REPO.git $GIT_HOME/

The api call will return different results when the head changes, invalidating docker's cache.

If you're dealing with private repos you can use github's x-oauth-basic authentication scheme with a personal access token like so:

   ADD https://$ACCESS_TOKEN:[e-mail]/repos/$USER/$REPO/git/refs/heads/$BRANCH version.json

(thx @captnolimar for a suggested edit to clarify authentication)

Answer 4

Issue 1996 is not yet available, but you have the following workaround:

FROM foo
ARG CACHE_DATE=2016-01-01
RUN git clone ...

docker build --build-arg CACHE_DATE=$(date) ....

That would invalidate cache after the ARG CACHE_DATE line for every build.

Or:

ADD http://www.convert-unix-time.com/api?timestamp=now /tmp/bustcache
RUN git pull

That would also invalidate cache after this ADD line.

Similar idea:

Add ARG command to your Dockerfile:

# Dockerfile
# add this and below command will run without cache
ARG CACHEBUST=1

When you need to rebuild with selected cache, run it with --build-arg option

$ docker build -t your-image --build-arg CACHEBUST=$(date +%s) .

then only layer below ARG command in Dockerfile will rebuild.

Answer 5

For github private repos, you could also pass in your username and password:

RUN git clone -b$BRANCH https://$USER:$[e-mail]/$USER/$REPO.git $GIT_HOME/