solr - java heap space out of memory

问题:

Can any one help me to find out the error cause in tomcat. i m using solr in it. starting it works fine. later i will raise this error. after restarting it works fine. please help me to find out the error.

Mar 7, 2011 10:36:47 AM org.apache.solr.common.SolrException log SEVERE: java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1068) at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:418) at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:85) at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:169) at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:69) at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:54) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:228) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:212) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:634) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:445) at java.lang.Thread.run(Unknown Source) Caused by: java.lang.OutOfMemoryError: Java heap space

回答1:

Try to increase your memory

Either set the environment variable JAVA_OPTS="-Xms256m -Xmx500m" which means heap start size is 256 MB, max 512 MB. Or edit your catalina.bat or .sh and add this line there.

You have to play around with these values, some weeks ago I had an SOLR indexer job, about 2 GB of data, 700MB heap were required.

回答2:

You can try lowering the default batchSize JDBC connection param in your dataSource definition.
DataImportHandler is designed to stream row one-by-one. It passes a fetch size value (default: 500) to Statement#setFetchSize which some drivers do not honor.

If you're using MsSQL you can add dataSource param responseBuffering="adaptive".

See here for more info:
http://wiki.apache.org/solr/DataImportHandlerFaq

回答3:

start tomcat with an -Xmx1024m or some larger value to give tomcat more heap space.

回答4:

you should definitely look into tuning your JVM. to start with you should alter your JVM parameters to emit a Garbage Collection log and then use a tool to analyze that log. this analysis will help you to pick sane values for your heap, and it will also allow you to gauge whether or not you really have a memory leak or just a heap tuning issue. see this related question for some info on GC analysis tools:

Analyze GC logs for Sun Hotspots, JVM 6