Why is the 64bit JVM faster than the 32bit one?

Question

Recently I've been doing some benchmarking of the write performance of my company's database product, and I've found that simply switching to a 64bit JVM gives a consistent 20-30% performance increase.

I'm not allowed to go into much detail about our product, but basically it's a column-oriented DB, optimised for storing logs. The benchmark involves feeding it a few gigabytes of raw logs and timing how long it takes to analyse them and store them as structured data in the DB. The processing is very heavy on both CPU and I/O, although it's hard to say in what ratio.

A few notes about the setup:

Processor: Xeon E5640 2.66GHz (4 core) x 2
RAM: 24GB
Disk: 7200rpm, no RAID
OS: RHEL 6 64bit
Filesystem: Ext4
JVMs: 1.6.0_21 (32bit), 1.6.0_23 (64bit)
Max heap size (-Xmx): 512 MB (for both 32bit and 64bit JVMs)

Constants for both JVMs:

• Same OS (64bit RHEL)
• Same hardware (64bit CPU)
• Max heap size fixed to 512 MB (so the speed increase is not due to the 64bit JVM using a larger heap)

For simplicity I've turned off all multithreading options in our product, so pretty much all processing is happening in a single-threaded manner. (When I turned on multi-threading, of course the system got faster, but the ratio between 32bit and 64bit performance stayed about the same.)

So, my question is... Why would I see a 20-30% speed improvement when using a 64bit JVM? Has anybody seen similar results before?

My intuition up until now has been as follows:

• 64bit pointers are bigger, so the L1 and L2 caches overflow more easily, so performance on the 64bit JVM is worse.

• The JVM uses some fancy pointer compression tricks to alleviate the above problem as much as possible. Details on the Sun site here.

• The JVM is allowed to use more registers when running in 64bit mode, which speeds things up slightly.

Given the above three points, I would expect 64bit performance to be slightly slower, or approximately equal to, the 32bit JVM.

Any ideas? Thanks in advance.

Edit: Clarified some points about the benchmark environment.

Answer 1

My best guess, based on a quick google for 32- vs 64-bit performance charts, is that 64 bit I/O is more efficient. I suppose you do a lot of I/O...

If memcpy is involved when moving the data, it's probably more efficient to copy longs than ints.

Answer 2

From: http://www.oracle.com/technetwork/java/hotspotfaq-138619.html#64bit_performance

"Generally, the benefits of being able to address larger amounts of memory come with a small performance loss in 64-bit VMs versus running the same application on a 32-bit VM. This is due to the fact that every native pointer in the system takes up 8 bytes instead of 4. The loading of this extra data has an impact on memory usage which translates to slightly slower execution depending on how many pointers get loaded during the execution of your Java program. The good news is that with AMD64 and EM64T platforms running in 64-bit mode, the Java VM gets some additional registers which it can use to generate more efficient native instruction sequences. These extra registers increase performance to the point where there is often no performance loss at all when comparing 32 to 64-bit execution speed.
The performance difference comparing an application running on a 64-bit platform versus a 32-bit platform on SPARC is on the order of 10-20% degradation when you move to a 64-bit VM. On AMD64 and EM64T platforms this difference ranges from 0-15% depending on the amount of pointer accessing your application performs."

Answer 3

Realize that the 64-bit JVM is not magic pixie dust that makes Java apps go faster. The 64-bit JVM allows heaps >> 4 GB and, as such, only makes sense for applications which can take advantage of huge memory on systems which have it.

Generally there is either a slight improvement (due to certain hardware optimizations on certain platforms) or minor degradation (due to increased pointer size). Generally speaking there will be a need for fewer GC's -- but when they do occur they will likely be longer.

In memory databases or search engines that can use the increased memory for caching objects and thus avoid IPC or disk accesses will see the biggest application level improvements. In addition a 64-bit JVM will also allow you to run many, many more threads than a 32-bit one, because there's more address space for things like thread stacks, etc. The maximum number of threads generally for a 32-bit JVM is ~1000but ~100000 threads with a 64-bit JVM.

Some drawbacks though:
Additional issues with the 64-bit JVM are that certain client oriented features like Java Plug-in and Java Web Start are not supported. Also any native code would also need to be compatible (e.g. JNI for things like Type II JDBC drivers). This is a bonus for pure-Java developers as pure apps should just run out of the box.

More on this Thread at Java.net

Answer 4

Without knowing your hardware I'm just taking some wild stabs

• Your specific CPU may be using microcode to 'emulate' some x86 instructions -- most notably the x87 ISA
• x64 uses sse math instead of x87 math, I've noticed a %10-%20 speedup of some math-heavy C++ apps in this case. Math differences could be the real killer if you're using strictfp.
• Memory. 64 bits gives you much more address space. Maybe the GC is a little less agressive on 64 bits mode because you have extra RAM.
• Is your OS is in 64b mode and running a 32b jvm via some wrapper utility?

Answer 5

The 64-bit instruction set has 8 more registers, this should make the code faster overall.

But, since processsors nowaday mostly wait for memory or disk, i suppose that either the memory subsystem or the disk i/o might be more efficient in 64-bit mode.