Getting out of memory error when restarting WildFl

I have a issue with WildFly AS on OpenShift. When i try to deploy WAR on server i keep getting following exception.

2014-12-20 04:31:33,072 INFO  [org.jboss.as.connector.deployment] (MSC service thread 1-7) JBAS010406: Registered connection factory java:/JmsXA
2014-12-20 04:31:33,444 ERROR [org.hornetq.core.client] (Thread-0 (HornetQ-client-global-threads-32821528)) HQ214017: Caught unexpected Throwable: java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method) [rt.jar:1.8.0_05]
        at java.lang.Thread.start(Thread.java:714) [rt.jar:1.8.0_05]
        at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950) [rt.jar:1.8.0_05]
        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368) [rt.jar:1.8.0_05]
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$Channel0Handler.notifyTopologyChange(ClientSessionFactoryImpl.java:1618) [hornetq-core-client-2.4.5.Final.jar:]
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$Channel0Handler.handlePacket(ClientSessionFactoryImpl.java:1609) [hornetq-core-client-2.4.5.Final.jar:]
        at org.hornetq.core.protocol.core.impl.ChannelImpl.handlePacket(ChannelImpl.java:641) [hornetq-core-client-2.4.5.Final.jar:]
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:556) [hornetq-core-client-2.4.5.Final.jar:]
        at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:532) [hornetq-core-client-2.4.5.Final.jar:]
        at org.hornetq.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1712) [hornetq-core-client-2.4.5.Final.jar:]
        at org.hornetq.core.remoting.impl.invm.InVMConnection$1.run(InVMConnection.java:165) [hornetq-server-2.4.5.Final.jar:]
        at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:104) [hornetq-core-client-2.4.5.Final.jar:]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_05]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_05]
        at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_05]

I don't know if its related but i think errors started when i added hibernate to my application. Here is the hibernate.cfg.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE hibernate-configuration PUBLIC "-//Hibernate/Hibernate Configuration DTD 3.0//EN" "http://www.hibernate.org/dtd/hibernate-configuration-3.0.dtd">
<hibernate-configuration>
    <session-factory>

        <property name="hibernate.dialect">org.hibernate.dialect.MySQLDialect</property>   
        <property name="connection.datasource">java:comp/env/jdbc/MyDataSource</property> 
        <property name="hibernate.current_session_context_class">thread</property>

        <!-- GENERAL CONFIGURATION -->
        <property name="connection.pool_size">1</property>
        <property name="hibernate.show_sql">false</property>
        <property name="hibernate.connection.release_mode">after_statement</property>
        <property name="hibernate.connection.autocommit">false</property>
        <property name="hibernate.connection.defaultAutoCommit">false</property>

        <!-- DB MAPING -->

    </session-factory>
</hibernate-configuration>

My HibernateUtil.java:

import org.hibernate.SessionFactory;
import org.hibernate.boot.registry.StandardServiceRegistryBuilder;
import org.hibernate.cfg.Configuration;
import org.hibernate.service.ServiceRegistry;

public class HibernateUtil {
    private static SessionFactory sessionFactory;

    public static SessionFactory getSessionFactory() {

        if (sessionFactory == null) {
            // loads configuration and mappings
            Configuration configuration = new Configuration().configure();
            ServiceRegistry serviceRegistry
                = new StandardServiceRegistryBuilder()
                    .applySettings(configuration.getProperties()).build();

            // builds a session factory from the service registry
            sessionFactory = configuration.buildSessionFactory(serviceRegistry);           
        }

        return sessionFactory;
    }
}

And part of my DaoImpl i'm using for login:

  public User login(String username, String password) {

        User result = null;

        Session session = HibernateUtil.getSessionFactory().openSession();
        try {
            String sql = "select s from User s where username=:username and password=password(:password)";
            Query query = session.createQuery(sql);
            query.setString("username", username);
            query.setString("password", password);
            result = (User) query.uniqueResult();
        }
        finally {
            session.close();
        }

        return result;
    }

I'm not 100% shure its problem with hibernate because i'we connected over SSH removed all in wildfly/standalone/deployments/ and restarted app but i'm still getting floowing error when i execute gear start command.

2014-12-20 04:46:47,792 WARN  [org.hornetq.core.server] (hornetq-failure-check-thread) HQ222082: error on connection failure check: java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method) [rt.jar:1.8.0_05]
        at java.lang.Thread.start(Thread.java:714) [rt.jar:1.8.0_05]
        at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950) [rt.jar:1.8.0_05]
        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1357) [rt.jar:1.8.0_05]
        at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor.execute(OrderedExecutorFactory.java:133) [hornetq-core-client-2.4.5.Final.jar:]
        at org.hornetq.core.remoting.server.impl.RemotingServiceImpl$FailureCheckAndFlushThread.run(RemotingServiceImpl.java:739) [hornetq-server-2.4.5.Final.jar:]

2014-12-20 04:46:48,674 ERROR [stderr] (Thread-0 (HornetQ-scheduled-threads-30017644)) Exception in thread "Thread-0 (HornetQ-scheduled-threads-30017644)" java.lang.OutOfMemoryError: unable to create new native thread
2014-12-20 04:46:48,675 ERROR [stderr] (Thread-0 (HornetQ-scheduled-threads-30017644))  at java.lang.Thread.start0(Native Method)
2014-12-20 04:46:48,675 ERROR [stderr] (Thread-0 (HornetQ-scheduled-threads-30017644))  at java.lang.Thread.start(Thread.java:714)
2014-12-20 04:46:48,675 ERROR [stderr] (Thread-0 (HornetQ-scheduled-threads-30017644))  at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
2014-12-20 04:46:48,676 ERROR [stderr] (Thread-0 (HornetQ-scheduled-threads-30017644))  at java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:1018)
2014-12-20 04:46:48,676 ERROR [stderr] (Thread-0 (HornetQ-scheduled-threads-30017644))  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160)
2014-12-20 04:46:48,676 ERROR [stderr] (Thread-0 (HornetQ-scheduled-threads-30017644))  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
2014-12-20 04:46:48,677 ERROR [stderr] (Thread-0 (HornetQ-scheduled-threads-30017644))  at java.lang.Thread.run(Thread.java:745)
2014-12-20 04:46:49,675 ERROR [stderr] (Thread-2 (HornetQ-scheduled-threads-30017644)) Exception in thread "Thread-2 (HornetQ-scheduled-threads-30017644)" java.lang.OutOfMemoryError: unable to create new native thread

回答1:

The small gears have a thread limit of 250 threads, try to limit the # of threads for the messaging service to 50 (or even try 20 depending on the app). The config is in standalone.xml file.

Eg:

<subsystem xmlns="urn:jboss:domain:messaging:2.0">
    <hornetq-server>
        <thread-pool-max-size>50</thread-pool-max-size>
        <scheduled-thread-pool-max-size>50</scheduled-thread-pool-max-size>     

        <!-- ... -->

    </hornetq-server>
</subsystem>

I hope this help.

回答2:

Check out this thread.

I found out that I had to disable not only messaging but also clustering.jgroups. However, I encountered OutOfMemoryError occasionally after that. I limited the db connection pool. By default it is unlimited.

Sadly this was not enough and I also went down with