Zero downtime on Heroku

Question

Is it possible to do something like the Github zero downtime deploy on Heroku using Unicorn on the Cedar stack?

I'm not entirely sure how the restart works on Heroku and what control we have over restarting processes, but I like the possibility of zero downtime deploys and up until now, from what I've read, it's not possible

There are a few things that would be required for this to work.

1. First off, we'd need backwards compatible migrations. I leave that up to our team to figure out.
2. Secondly, we'd want to migrate the db right after a push, but before the restart (assuming our migrations are fully backwards compatible, this should not affect anything)
3. Thirdly, we'd want to instruct Unicorn to launch a new master process and fork some workers, then swap the PIDs and gracefully shut down the old process/workers

I've scoured the docs but I can't find anything that would indicate this is possible on Heroku. Any thoughts?

Answer 1

It is possible, but requires a fair amount of forward planning. As of Rails 3.1 there's three tasks that need carrying out

• Upload the new code
• Run any database migrations
• Sync the assets

Uploading code and restarting is fairly straightforward, the main problem lies with the other two, but the way round them is the pretty much the same.

Essentially you need to:

• Make the code compatible with the migration you need to run
• Run the migration, and remove any code written specifically for it

For instance, if you want to remove a column, you’ll need to deploy a patch telling ActiveRecord to ignore it first. Only then you can deploy the migration, and clean up that patch.

In short, you need to consider your database and the code compatability an work around them so that the two can overlap in terms of versioning.

An alternative to this method might be to have two versions of the application running on Heroku at the same time. When you deploy, switch the domain to the other version, do the deploy, and switch it back again. This will help in most instances, but again, database compat is an issue.

Personally, I would say that if your deployments are significant to require this sort of consideration, taking parts of the application offline are probably the safest answer. By breaking up an application into several smaller applications can help mitigate this and is a mechanism that I use regularly.

Answer 2

You might be interested in their feature called preboot.

Taken from their documentation:

This feature provides seamless deploys by booting web dynos with new code before killing existing web dynos.

Some apps take a long time to boot up, and this can cause unacceptable delays in serving HTTP requests during deployment.

There are a few caveats:

• You must have at least two web dynos to use this feature. If you have your web process type scaled to 1 or 0, preboot will be disabled.
• Whoever is doing the deployment will have to wait a few minutes before the new code starts serving user requests; this happens later than it would without preboot (but in the meanwhile, user requests are still served promptly by old dynos).
• There will be a short period (a minute or two) where heroku ps shows the status of the new code, but user requests are still being served by old code.

There is much more information about it, so refer to their documentation.

Answer 3

No - this is currently not possible using Unicorn on Heroku cedar. I've been bugging Heroku about this for weeks.

Here was Heroku Support's reply to my email on March 8, 2012:

Hi, you could enable maintenance mode when doing a deploy, at least your users would see a maintenance page instead of an error, and also request queue wouldn't build up.

We're definitely aware this is a pain and we're working to offer rolling / zero-downtime deploys in the future. We have no ETA to announce, though.

Answer 4

I can't address migrations, but the part about restarting processes and avoiding wait time:

There is an beta feature for heroku called preboot. After a deploy, it boots your new dynos first and waits a while before switching traffic and killing the old ones:

https://devcenter.heroku.com/articles/labs-preboot/

I also wrote a blog post that has some measurements on my app's performance improvements using this feature:

http://ylan.segal-family.com/blog/2012/08/27/deploy-to-heroku-with-near-zero-downtime/