Best practices for deploying Java webapps with min

Question

When deploying a large Java webapp (>100 MB .war) I'm currently use the following deployment process:

• The application .war file is expanded locally on the development machine.
• The expanded application is rsync:ed from the development machine to the live environment.
• The app server in the live environment is restarted after the rsync. This step is not strictly needed, but I've found that restarting the application server on deployment avoids "java.lang.OutOfMemoryError: PermGen space" due to frequent class loading.

Good things about this approach:

• The rsync minimizes the amount of data sent from the development machine to the live environment. Uploading the entire .war file takes over ten minutes, whereas an rsync takes a couple of seconds.

Bad things about this approach:

• While the rsync is running the application context is restarted since the files are updated. Ideally the restart should happen after the rsync is complete, not when it is still running.
• The app server restart causes roughly two minutes of downtime.

I'd like to find a deployment process with the following properties:

• Minimal downtime during deployment process.
• Minimal time spent uploading the data.
• If the deployment process is app server specific, then the app server must be open-source.

Question:

• Given the stated requirements, what is the optimal deployment process?

Answer 1

Can't you make a local copy of the current web application on the web server, rsync to that directory and then perhaps even using symbolic links, in one "go", point Tomcat to a new deployment without much downtime?

Answer 2

Your approach to rsync the extracted war is pretty good, also the restart since I believe that a production server should not have hot-deployment enabled. So, the only downside is the downtime when you need to restart the server, right?

I assume all state of your application is hold in the database, so you have no problem with some users working on one app server instance while other users are on another app server instance. If so,

Run two app servers: Start up the second app server (which listens on other TCP ports) and deploy your application there. After deployment, update the Apache httpd's configuration (mod_jk or mod_proxy) to point to the second app server. Gracefully restarting the Apache httpd process. This way you will have no downtime and new users and requests are automatically redirected to the new app server.

If you can make use of the app server's clustering and session replication support, it will be even smooth for users which are currently logged in, as the second app server will resync as soon as it starts. Then, when there are no accesses to the first server, shut it down.

Answer 3

My advice is to use rsync with exploded versions but deploy a war file.

1. Create temporary folder in the live environment where you'll have exploded version of webapp.
2. Rsync exploded versions.
3. After successfull rsync create a war file in temporary folder in the live environment machine.
4. Replace old war in the server deploy directory with new one from temporary folder.

Replacing old war with new one is recommended in JBoss container (which is based on Tomcat) beacause it'a atomic and fast operation and it's sure that when deployer will start entire application will be in deployed state.

Answer 4

Hot Deploy a Java EAR to Minimize or Eliminate Downtime of an Application on a Server or How to “hot” deploy war dependency in Jboss using Jboss Tools Eclipse plugin might have some options for you.

Deploying to a cluster with no downtime is interesting too.

JavaRebel has hot-code deployement too.

Answer 5

If static files are a big part of your big WAR (100Mo is pretty big), then putting them outside the WAR and deploying them on a web server (e.g. Apache) in front of your application server might speed up things. On top of that, Apache usually does a better job at serving static files than a servlet engine does (even if most of them made significant progress in that area).

So, instead of producing a big fat WAR, put it on diet and produce:

• a big fat ZIP with static files for Apache
• a less fat WAR for the servlet engine.

Optionally, go further in the process of making the WAR thinner: if possible, deploy Grails and other JARs that don't change frequently (which is likely the case of most of them) at the application server level.

If you succeed in producing a lighter WAR, I wouldn't bother of rsyncing directories rather than archives.

Strengths of this approach:

1. The static files can be hot "deployed" on Apache (e.g. use a symbolic link pointing on the current directory, unzip the new files, update the symlink and voilà).
2. The WAR will be thinner and it will take less time to deploy it.

Weakness of this approach:

1. There is one more server (the web server) so this add (a bit) more complexity.
2. You'll need to change the build scripts (not a big deal IMO).
3. You'll need to change the rsync logic.

Answer 6

Not a "best practice" but something I just thought of.

How about deploying the webapp through a DVCS such as git?

This way you can let git figure out which files to transfer to the server. You also have a nice way to back out of it if it turns out to be busted, just do a revert!