Best CD strategy for Kubernetes Deployments

Question

Our current CI deployment phase works like this:

1. Build the containers.
2. Tag the images as "latest" and < commit hash >.
3. Push images to repository.
4. Invoke rolling update on appropriate RC(s).

This has been working great for RC based deployments, but now that the Deployment object is becoming more stable and an underlying feature, we want to take advantage of this abstraction over our current deployment schemes and development phases.

What I'm having trouble with is finding a sane way to automate the update of a Deployment with the CI workflow. What I've been experimenting with is splitting up the git repo's and doing something like:

1. [App Build] Build the containers.
2. [App Build] Tag the images as "latest" and < commit hash >.
3. [App Build] Push images to repository.
4. [App Build] Invoke build of the app's Deployment repo, passing through the current commit hash.
5. [Deployment Build] Interpolate manifest file tokens (currently just the passed commit hash e.g. image: app-%%COMMIT_HASH%%)
6. [Deployment Build] Apply the updated manifest to the appropriate Deployment resource(s).

Surely though there's a better way to handle this. It would be great if the Deployment monitored for hash changes of the image's "latest" tag...maybe it already does? I haven't had success with this. Any thoughts or insights on how to better handle the deployment of Deployment would be appreciated :)

Answer 1

The Deployment only monitors for pod template (.spec.template) changes. If the image name didn't change, the Deployment won't do the update. You can trigger the rolling update (with Deployments) by changing the pod template, for example, label it with commit hash. Also, you'll need to set .spec.template.spec.containers.imagePullPolicy to Always (it's set to Always by default if :latest tag is specified and cannot be update), otherwise the image will be reused.

Answer 2

We've been practising what we call GitOps for a while now.

What we have is a reconciliation operator, which connect a cluster to configuration repository and makes sure that whatever Kubernetes resources (including CRDs) it finds in that repository, are applied to the cluster. It allows for ad-hoc deployment, but any ad-hoc changes to something that is defined in git will get undone in the next reconciliation cycle. The operator is also able to watch any image registry for new tags, an update image attributes of Deployment, DaemonSet and StatefulSet types of objects. It makes a change in git first, then applies it to the cluster.

So what you need to do in CI is this:

1. Build the containers.
2. Tag the images as <commit_hash>.
3. Push images to repository.

The agent will take care of the rest for you, as long you've connected it to the right config repo where the app Deployment object can be found.

For a high-level overview, see:

• Google Cloud Platform and Kubernetes
• Deploy Applications & Manage Releases

---

Disclaimer: I am a Kubernetes contributor and Weaveworks employee. We build open-source and commercial tools that help people to get to production with Kubernetes sooner.