How to mimic '--volumes-from' in Kubernete

问题:

I'm looking for a pattern that allows to share volumes between two containers running on the same pod in Kubernetes.

My use case is: I have a Ruby on Rails application running inside a docker container. The docker image contains static assets in /app/<app-name>/public directory, and I need to access those assets from the nginx container running alongside in the same pod.

In 'vanilla' docker I would have used --volumes-from flag to share this directory:

docker run --name app -v /app/<app-dir>/public <app-image>
docker run --volumes-from app nginx

After reading this doc: https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/volumes.md I tried this (only relevant entries presented):

spec:
  containers:
    - image: <app-image>
      name: <app-name>
      volumeMounts:
        - mountPath: /app/<app-name>/public
          name: assets
    - image: nginx
      name: nginx
      volumeMounts:
        - mountPath: /var/www/html
          name: assets
          readOnly: true
    volumes:
      - name: assets
        hostPath:
          path: /tmp/assets

But:

• Even though /tmp/assets on the node exists, it's empty
• /app/<app-name>/public inside the app container is also empty

As a workaround I'm gonna try to populate the shared directory when the application container is up (simply cp /app/<app-name>/public/* to shared directory), but I really dislike this idea.

Question: how to mimic --volumes-from in Kubernetes, or if there is no direct counterpart, how can I share files from one container to other running in the same pod ?

apiVersion: v1beta3

Client Version: version.Info{Major:"0", Minor:"17", GitVersion:"v0.17.0", GitCommit:"82f8bdac06ddfacf493a9ed0fedc85f5ea62ebd5", GitTreeState:"clean"}
Server Version: version.Info{Major:"0", Minor:"17", GitVersion:"v0.17.0", GitCommit:"82f8bdac06ddfacf493a9ed0fedc85f5ea62ebd5", GitTreeState:"clean"}

回答1:

[update-2016-8] In latest Kubernetes release, you can use a very nice feature named init-container to replace the postStart part in my answer below, which will make sure the container order.

NOTE: initContainer is still a beta feature so the work version of this yaml is actually like: http://kubernetes.io/docs/user-guide/production-pods/#handling-initialization, please notice the pod.beta.kubernetes.io/init-containers part.

---original answer begin---

Actually, you can. You need to use container life cycle handler to control what files/dirs you want to share with other containers. Like:

---
apiVersion: v1
kind: Pod
metadata:
    name: server
spec:
    restartPolicy: OnFailure
    containers:
    - image: resouer/sample:v2
      name: war
      lifecycle:
        postStart:
          exec:
            command:
              - "cp"
              - "/sample.war"
              - "/app"
      volumeMounts:
      - mountPath: /app
        name: hostv1 
    - name: peer
      image: busybox
      command: ["tail", "-f", "/dev/null"]
      volumeMounts:
      - name: hostv2
        mountPath: /app/sample.war
    volumes:
    - name: hostv1
      hostPath:
          path: /tmp
    - name: hostv2
      hostPath:
          path: /tmp/sample.war

Please check my gist for more details:

https://gist.github.com/resouer/378bcdaef1d9601ed6aa

And of course you can use emptyDir. Thus, war container can share its /sample.war to peer container without mess peer's /app directory.

If we can tolerate /app been overridden, it will be much simpler:

---
apiVersion: v1
kind: Pod
metadata:
  name: javaweb-2
spec:
  restartPolicy: OnFailure
  containers:
  - image: resouer/sample:v2
    name: war
    lifecycle:
      postStart:
        exec:
          command:
            - "cp"
            - "/sample.war"
            - "/app"
    volumeMounts:
    - mountPath: /app
      name: app-volume
  - image: resouer/mytomcat:7.0
    name: tomcat
    command: ["sh","-c","/root/apache-tomcat-7.0.42-v2/bin/start.sh"]
    volumeMounts:
    - mountPath: /root/apache-tomcat-7.0.42-v2/webapps
      name: app-volume
    ports:
    - containerPort: 8080
      hostPort: 8001 
  volumes:
  - name: app-volume
    emptyDir: {}

回答2:

The answer is - for now - you can't. Here's a couple of discussion threads from the Kubernetes issues:

• https://github.com/GoogleCloudPlatform/kubernetes/issues/6120
• https://github.com/GoogleCloudPlatform/kubernetes/issues/831

However, may I suggest that you have an alternate design that might work better?

1. If your assets are locked at the point of the container going live, you could use something like gitRepo volume which would copy it to an emptyDir at the point of going live, and would mean you wouldn't have to move the content around at all, just download it directly to the shared directory.
2. If your assets are locked at the point of the container being built, it's probably best to copy them in at that point, using the Docker COPY command.
3. If you really want to stick with the way you're doing it, you would have to copy the content to the emptyDir volume, which is designed for exactly what you're looking for (minus the lack of having to copy it in).

NFS[1] volumes also could solve your problem, but may be overly complex.

Additionally, I'd recommend that these two services exist in different pods, so you can scale each separately. You can create a service endpoint to communicate between them if you need to.

[1] https://github.com/GoogleCloudPlatform/kubernetes/blob/master/examples/nfs/nfs-web-pod.yaml

回答3:

Further update from the future:

There is now a FlexVol plugin for Docker volumes: https://github.com/dims/docker-flexvol

At the time of writing, FlexVol is still an alpha feature though, so caveat emptor.

回答4:

If you are using Docker v17.0.5 or greater you can use a multi-stage build to copy files from one of your containers to the other during build time. This is a great primer on the advanced features at https://medium.com/@tonistiigi/advanced-multi-stage-build-patterns-6f741b852fae

The way I used it to copy static assets from my backend container into Nginx proxy is

ARG API_BACKEND_CONTAINER="api:backend"
FROM $API_BACKEND_CONTAINER as source

FROM nginx:mainline-alpine

ARG NGINX_ROOT=/usr/share/nginx/html/
COPY --from=source  /var/share/api/static/ ${NGINX_ROOT}

The great thing is that because the API_BACKEND_CONTAINER is a build arg I'm able to pass in the tag of the latest API build.