Best way to deploy large *.war to tomcat

Question

During development I frequently have to deploy a large war-file (~45 MB) to a remote test server, normally I copy the file with scp to the server.

The WEB-INF/lib folder makes up the largest part of the war file, which includes all the required libraries (spring, apache-cxf, hibernate,...).

Now I'm searching for an fast and easy a way to redeploy only my altered files.

And how can I determine which packages are really needed by the webapp, because spring and apache-cxf comes with a lot of libs, I'm sure I don't need all of them.

Answer 1

I use rsync to copy my .war from my local machine to production. It usually provides a big speed up, around 8-10 times.

Another option is to use git to store the .war files. When you git push a new .war, only the differences are transferred. Also a big speed up. Some people say that git is not designed to store big files, it gets slow and does not work very well. In fact yes, the repo will grow a lot, but in some cases it could be a good option.

Some numbers: My .war is about 50MB and when I deploy a new version, it only copies about ~4MB instead of uploading a complete new war. Both with git and rsync.

UPDATE: The problem I ran into it's the git repo cannot be cloned after it has several .war versions because it will take forever to create all the deltas and transmit them to the client.

I changed the strategy by uploading the .war files into dropbox. Dropbox also uses kind of rsync and only copies deltas. From the server I wget the .war and re-reploy the app. Hope this helps.

Answer 2

I don't think there's a faster way to redeploy only the changes to a WAR file.

If you deploy in exploded fashion you can see what file timestamps have changed and act accordingly, but you'll have to write code to do it.

I don't know if OSGi can be a help here. That would allow you to partition your problem into modules that are more independent and swap-able.

Just curious:

1. How long does it take now?
2. Do you use continuous integration to build and deploy?

Answer 3

When you deploy a .war, the first thing Tomcat does is to unpack that file into its webapps directory, in a subdirectory with the same name as your .war.

During development, you obviously have access to your .class files, the .jar files, configuration files and whatever else eventually goes into your .war. You can easily establish a small subset of files affected by your changes. Figure that out, and then use a script or an ant task or whatever to copy just that small handful of files straight into the webapps/yourapp directory on the server.

To see your changes take effect, you'll need to re-start your application. If Tomcat is in development mode, one easy way to force a reload (and restart, of course) is to update WEB-INF/web.xml. So have your deployment process touch that file or otherwise update it in a way that will give it a new timestamp, scp that over too (preferrably as the last of the files you update) and you should have a quick and easy reload.

Answer 4

What I do is exclude WEB-INF/lib/*.jar files from the WAR and reassemble on the server side. In my case, this trims a 60MB WAR down to 250k which allows for really fast deployment.

The <exclude name="**/lib/*.jar"/> command is what excludes the jar's (see last code snippet for ANT build)

On the server side, it's quite easy to assemble a fully populated WAR from the trimmed WAR:

1. unzip/explode the trimmed WAR that was created by the ANT script below
2. copy the server repository jar files into the exploded WEB-INF/lib
3. zip is all up into a new (large) WAR.
4. deploy as usual.

For example:

unzip ../myapp.trimmed.war
mkdir WEB-INF/lib
cp ../war_lib_repository/* WEB-INF/lib
zip -r ../myapp.war .

Maybe not the most elegant solution, but it saves time on frequent deployment of large WAR's. I'd like to be able to do this with Maven so if anyone has suggestions, please let me know.

ANT build.xml:

<property file="build.properties"/>
<property name="war.name" value="myapp.trimmedwar"/>
<property name="deploy.path" value="deploy"/>   
<property name="src.dir" value="src"/>
<property name="config.dir" value="config"/>
<property name="web.dir" value="WebContent"/>
<property name="build.dir" value="${web.dir}/WEB-INF/classes"/>
<property name="name" value="${war.name}"/>

<path id="master-classpath">
    <fileset dir="${web.dir}/WEB-INF/lib">
        <include name="*.jar"/>         
    </fileset>
    <!-- other classes to include -->
    <fileset dir="${birt.runtime}/ReportEngine/lib">
        <include name="*.jar"/>
    </fileset>
    <pathelement path="${build.dir}"/>
</path> 

<target name="build" description="Compile main source tree java files">
    <mkdir dir="${build.dir}"/>
    <javac destdir="${build.dir}" debug="true" deprecation="false" optimize="false" failonerror="true">
        <src path="${src.dir}"/>
        <classpath refid="master-classpath"/>
    </javac>
</target>

<target name="createwar" depends="build" description="Create a trimmed WAR file (/lib/*.jar) excluded for size">
    <!-- copy the hibernate config file -->
    <copy todir="${web.dir}/WEB-INF/classes">
        <!-- copy hibernate configs -->
        <fileset dir="${src.dir}/" includes="**/*.cfg.xml" />
    </copy>     
    <copy todir="${web.dir}/WEB-INF/classes">
        <fileset dir="${src.dir}/" includes="**/*.properties" />
    </copy>             
    <!-- copy hibernate classes -->
    <copy todir="${web.dir}/WEB-INF/classes" >
        <fileset dir="${src.dir}/" includes="**/*.hbm.xml" />
    </copy>
    <war destfile="${name}.war" webxml="${web.dir}/WEB-INF/web.xml">
        <fileset dir="${web.dir}">
            <include name="**/*.*"/>
            <!-- exlude the jdbc connector because it's on the server's /lib/common -->
            <exclude name="**/mysql-connector*.jar"/>
            <!-- exclude these jars because they're already on the server (will be wrapped into the trimmed war at the server) -->
            <exclude name="**/lib/*.jar"/>
        </fileset>
    </war>
    <copy todir="${deploy.path}" preservelastmodified="true">
        <fileset dir=".">
            <include name="*.war"/>
        </fileset>
    </copy>
</target>           

Answer 5

Improving on Rori Stumpf answer, here is a Gradle task for the 'thinning'

war {
    archiveName "v1.war"
}

task createThinWar(type: Copy) {
    dependsOn 'war'

    def tmpFolder = "${buildDir}/tmp/thin"
    def outputDir = "${buildDir}/libs"

    // Extract the war (zip) contents
    from zipTree("${outputDir}/v1.war")
    into "${tmpFolder}/v1"

    doLast {
        // Extracting the war third party libraries to a separate dir
        ant.move(file: "${tmpFolder}/v1/WEB-INF/lib", tofile: "${tmpFolder}/v1-libs")

        // Zip the third party libraries dir
        ant.zip(destfile: "${outputDir}/v1-libs.zip") {
            fileset(dir: "${tmpFolder}/v1-libs")
        }

        // Finally zip the thinned war back
        ant.zip(destfile: "${outputDir}/v1-thin.war") {
            fileset(dir: "${tmpFolder}/v1")
        }
    }
}

This will generate v1-thin.war (weighting less then 1mb), and a libs zip.

deploy the thin war to the server (and reconstruct the libs there), and deploy the libs zip whenever you modified the versions/add libraries.