Configure Eclipse to pre-bundle App Engine classes

Question

After some discussion with a colleague from another company that also uses App Engine, he told me that he managed to cut down his app warm up time from ~15 seconds to ~5 seconds using the following steps:

1. Configure Eclipse to bundle classes produced during compilation into a single JAR file.
2. Configure Eclipse to upload this single JAR file instead of hundreds (or thousands) of separate Java classes during App Engine deployment.

He argues that during instance warm up, since the instance need to load only a single bundled JAR file instead of thousands of separate classes, the warm up would be significantly faster. Any thoughts or opinions about this?

I would definitely like to try it by myself, but I don't have enough Eclipse-mojo to know how to configure such steps. Does anyone know how to configure Eclipse or the Google Plugin for Eclipse to do the steps outlined above? (And have the deployed apps successfully runs in App Engine, of course)

Thank You,

Answer 1

As stated in an earlier answer, the App Engine SDK supports packaging WEB-INF/classes into a jar file, which will end up in WEB-INF/lib/_ah_webinf_classes-0000.jar. You can activate this

1. using the appcfg tool with the option --enable_jar_classes.

2. using the Google Plugin for Eclipse by configuring the properties of either your WAR or EAR project: Project properties > Google App Engine > Deployment > "Package WEB-INF/classes as a jar"

For me, on App Engine 1.9.4, this resulted in only a minor improvement in instance spin-up (about 5-10 %, if any).

---

Note that this will package all files in WEB-INF/classes (not only .class ones). Following the change, I got an error message during instantiation about not being able to read the logging.properties file anymore; probably because the new jar file hasn't been read at that time:

Unable to read the java.util.logging configuration file, WEB-INF/classes/logging.properties

As a workaround, I changed the path in appengine-web.xml to WEB-INF/logging.properties and configured the maven-war-plugin to copy the file to that location:

                <webResources>
                    <resource>
                        <directory>lib</directory>
                        <targetPath>WEB-INF/lib</targetPath>
                    </resource>
                    <resource>
                        <!-- Workaround: During GAE deployment, all files in WEB-INF/classes will be packaged into WEB-INF/lib/_ah_webinf_classes-0000.jar, 
                            which prevents the logging.properties referenced in appengine-web.xml from being read. -->
                        <directory>src/main/resources</directory>
                        <includes>
                            <include>logging.properties</include>
                        </includes>
                        <targetPath>WEB-INF</targetPath>
                    </resource>
                </webResources>

Answer 2

I think the easiest way to do this would be to have a separate project that you write all the code in as well as store any other dependencies. When you're ready to deploy, Eclipse's export to runnable JAR file can repack all the dependencies and your classes into a single JAR file, which you would then use as your one dependency in your deployable Google App Engine project which would basically consist of any config files in the WEB-INF folder as well as other static resources + the one big JAR file.

Alternatively to the eclipse export, you can set that up as a Maven or Ant build which can also do the repacking.

Answer 3

One way this can be achieved if by doing the deployment through Ant, as described in: https://developers.google.com/appengine/docs/java/tools/ant

Next you can modify the ant build.xml file to call the ant command for building the jar file. Just before the actual deployment you can either delete or move the compiled artifacts away. The build jar-file should be placed in the WAR/WEB-INF/lib folder.

Drawback of this solution is that you have to deploy through the build.xml, i.s.o. through the appengine eclipse plugin.

Answer 4

He argues that during instance warm up, since the instance need to load only a single bundled WAR file instead of thousands of separate classes, the warm up would be significantly faster. Any thoughts or opinions about this?

I doubt it. WAR is just a ZIP file, which gets unpacked when it's deployed on the server. So there is an additional step, meaning the process can be equally fast (if unpacked when uploaded) or slower (if unpacked when instance is spun up).

Answer 5

I don't know how (or if) you can integrate it into eclipse, but it's fairly trivial to do with ant:

<import file="${appengine.sdk.dir}/config/user/ant-macros.xml" />

<target name="deploy">
    <delete dir="${staging.dir}" />
    <mkdir dir="${staging.dir}" />

    <copy todir="${staging.dir}">
        <fileset dir="war">
            <exclude name="WEB-INF/classes/**" />
            <exclude name="WEB-INF/appengine-generated/**" />
        </fileset>
    </copy>
    <jar destfile="${staging.dir}/WEB-INF/lib/classes.jar" basedir="${classes.dir}" />

    <appcfg action="update" war="${staging.dir}" />
</target>

I will add that I did not experience a 3X reduction in app startup time. I posted some experimental numbers in this thread:

https://groups.google.com/d/msg/google-appengine/dStBW4wIemY/K69f9ufDiN0J

What I found is that instead of varying wildly from 20-45s, it made my app consistently load in 20s. It has not subsequently remained this consistent, but I still jar my classes as a standard part of deployment now.

Answer 6

What we did in Eclipse was:

• In Project Properties > Java Build Path (Source tab) change output folder from war/WEB-INF/classes to something else, e.g. bin (I believe Eclipse complained about this once)
• In Project Properties > Builders add a new builder (I used type "program builder" and pointed it to the jar executable in my JDK, but as others mentioned an Ant builder would work too). Obviously you should configure this builder to take its input from wherever you decided to write your class files to, and output to something like war/WEB-INF/lib/myclasses.jar

You can configure the jar builder to execute automatically as class files change (which in turn are usually automatically recompiled as your source files change).

There are some downsides, though. For some reason the google eclipse plugin gets confused by your changing the output directory of the java compiler. This means you will have to manually point to the war directory when deploying, and I believe you'll have to manually copy some GAE jars into the war/WEB-INF/lib folder.