Outof memory error while working on large dataset

Question

I am running a code on LSI, which requires first fetching a lot of data from database. It is working fine for small data-set. As, i increase the data-set, it gives me the following error.

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

I am currently running the code on system having 2 GB of RAM. Is the error related to RAM capacity or due to something else.

Thanks!

Answer 1

If its related to your RAM your processor will stop Java. So definitly not. Its because java heap memory is full and all the objects in Heap space is active so garbage collection process will not carried out. so there is not enough memmory in heap space for processing further data. Bcoz of this reason you get this error.

Answer 2

First check how much heap space you have assigned to your server. If it seems to low then you can increase it using following parameters -

-Xms128m   -  minimum heap size in MB
-Xmx1024m -  maximum heap size in MB

Generally we get this exception when available hep size gets full and no memory available for program normal execution.

Answer 3

Besides increasing the maximum heap size your application can have (may be all you need to do), it sounds like you might be able to significantly reduce the amount of memory you are using. If you are indexing documents, in a standard LSI set up, you should be able to load just 1 document into memory at a time. So say you have a list of documents, you loop over the list and: load, index, close. Rinse and repeat.

I'm thinking this might be your issue since your code works on smaller datasets. The other possibility is that when you expand your data set you are including some gigantic document that uses up all heap space you have. In that case the easiest solution is to just increase the heap space.

Answer 4

When you run Java you'll have to pass VM parameters for your specific concerns. You need to increase heap values:

-Xms40m - minimum heap size in MB
-Xmx1024m - maximum heap size in MB

java test.java -Xms40m -Xmx1024m

when launching your app. For more information refer Oracle documentation. Or if you use Eclipse increase this size in ecliplse.ini file.

Answer 5

1. -Xmx for maximum heap size,
2. -Xms for initial heap size,

The important point is above both syntax are case-sensitive. So careful.

How to set java heap size in Tomcat? Stop Tomcat server, set environment variable CATALINA_OPTS, and then restart Tomcat. Look at the file tomcat-install/bin/catalina.sh or catalina.bat for how this variable is used. For example,

set CATALINA_OPTS=-Xms512m -Xmx512m (Windows, no "" around the value) export CATALINA_OPTS="-Xms512m -Xmx512m" (ksh/bash, "" around the value) setenv CATALINA_OPTS "-Xms512m -Xmx512m" (tcsh/csh, "" around the value)

In catalina.bat or catallina.sh, you may have noticed CATALINA_OPTS, JAVA_OPTS, or both can be used to specify Tomcat JVM options. What is the difference between CATALINA_OPTS and JAVA_OPTS? The name CATALINA_OPTS is specific for Tomcat servlet container, whereas JAVA_OPTS may be used by other java applications (e.g., JBoss). Since environment variables are shared by all applications, we don't want Tomcat to inadvertently pick up the JVM options intended for other apps. I prefer to use CATALINA_OPTS.

How to set java heap size in JBoss?

Stop JBoss server, edit $JBOSS_HOME/bin/run.conf, and then restart JBoss server. You can change the line with JAVA_OPTS to something like:

JAVA_OPTS="-server -Xms128m -Xmx128m"

How to set java heap size in Eclipse? You have 2 options: 1. Edit eclipse-home/eclipse.ini to be something like the following and restart Eclipse.

-vmargs-Xms64m-Xmx256m2. Or, you can just run eclipse command with additional options at the very end. Anything after -vmargs will be treated as JVM options and passed directly to the JVM. JVM options specified in the command line this way will always override those in eclipse.ini. For example,

eclipse -vmargs -Xms64m -Xmx256m

How to set java heap size in NetBeans? Exit NetBeans, edit the file netbeans-install/etc/netbeans.conf. For example,

netbeans_default_options="-J-Xms512m -J-Xmx512m -J-XX:PermSize=32m -J-XX:MaxPermSize=128m -J-Xverify:none

If you have doubt please refer the link below. Thank them for provinding the details.

enter link description here