ActiveMQ JMS XA Atomikos - Transaction not started

The situation is we want to use XA transactions to coordinate activity between ActiveMQ and Hibernate (Sql Server 2008).

We are using:

Spring 3.0.5
Hibernate 3.6.2
ActiveMQ 5.5.0
Atomikos 3.7

We are seeing the following errors getting generated in the log file related to the transaction has not been started. These are always related to JMS.

Transaction '[ID]' has not been started.

These are getting generated into the logs all the time.

The issue is more complicated in that we have 3 web application that work with the same JMS queues and the errors do not seem to be generated when only a single web application is running.

These are deployed to separate instances of Tomcat 7.0.14 running on the same machine.

2011-05-31 15:04:27,065 [Atomikos:30] WARN  - [com.atomikos.diagnostics.Slf4jConsole] : XA resource 'XAJMS': rollback for XID '3139322E3136382E302E35332E746D30363636333030303031:3139322E3136382E302E35332E746D36363633' raised 0: unknown
javax.transaction.xa.XAException: Transaction 'XID:1096044365:3139322e3136382e302e35332e746d30363636333030303031:3139322e3136382e302e35332e746d36363633' has not been started.
    at org.apache.activemq.TransactionContext.toXAException(TransactionContext.java:732)
    at org.apache.activemq.TransactionContext.rollback(TransactionContext.java:497)
    at com.atomikos.datasource.xa.XAResourceTransaction.rollback(XAResourceTransaction.java:690)
    at com.atomikos.icatch.imp.RollbackMessage.send(RollbackMessage.java:72)
    at com.atomikos.icatch.imp.PropagationMessage.submit(PropagationMessage.java:111)
    at com.atomikos.icatch.imp.Propagator$PropagatorThread.run(Propagator.java:87)
    at com.atomikos.icatch.imp.Propagator.submitPropagationMessage(Propagator.java:66)
    at com.atomikos.icatch.imp.HeurHazardStateHandler.onTimeout(HeurHazardStateHandler.java:124)
    at com.atomikos.icatch.imp.CoordinatorImp.alarm(CoordinatorImp.java:1105)
    at com.atomikos.timing.PooledAlarmTimer.notifyListeners(PooledAlarmTimer.java:112)
    at com.atomikos.timing.PooledAlarmTimer.run(PooledAlarmTimer.java:99)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
Caused by: javax.transaction.xa.XAException: Transaction 'XID:1096044365:3139322e3136382e302e35332e746d30363636333030303031:3139322e3136382e302e35332e746d36363633' has not been started.
    at org.apache.activemq.broker.TransactionBroker.getTransaction(TransactionBroker.java:290)
    at org.apache.activemq.broker.TransactionBroker.rollbackTransaction(TransactionBroker.java:177)
    at org.apache.activemq.broker.MutableBrokerFilter.rollbackTransaction(MutableBrokerFilter.java:131)
    at org.apache.activemq.broker.TransportConnection.processRollbackTransaction(TransportConnection.java:436)
    at org.apache.activemq.command.TransactionInfo.visit(TransactionInfo.java:104)
    at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:306)
    at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:179)
    at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:69)
    at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
    at org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:227)
    at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
    at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:220)
    at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:202)
    at java.lang.Thread.run(Unknown Source)

Our Spring configuration:

<Resource name="jms/ConnectionFactory" 
    auth="Container" 
    type="org.apache.activemq.ActiveMQXAConnectionFactory" 
    description="JMS XA Connection Factory"
    factory="org.apache.activemq.jndi.JNDIReferenceFactory" 
    brokerURL="tcp://${activemq.server.name}:${activemq.server.port}" 
    brokerName="LocalActiveMQBroker" />


<bean id="atomikosConnectionFactory"  
      class="com.atomikos.jms.AtomikosConnectionFactoryBean"  
      init-method="init" destroy-method="close"> 
    <property name="uniqueResourceName" value="XAJMS" /> 
    <property name="xaConnectionFactory" ref="jmsConnectionFactory" />
    <property name="maxPoolSize" value="40" />
</bean>

<bean id="getSomeStuffListenerAdapter" 
      class="org.springframework.jms.listener.adapter.MessageListenerAdapter">
    <property name="delegate" ref="targetElement" />
    <property name="defaultListenerMethod" value="doStuff" />
    <property name="messageConverter" ref="myMessageConverter" />
</bean>

<bean id="getSomeStuffListenerContainer" 
      class="org.springframework.jms.listener.DefaultMessageListenerContainer"
      depends-on="txManager">
    <property name="connectionFactory" ref="atomikosConnectionFactory"/>
    <property name="destination" ref="jmsQueue01"/>
    <property name="messageListener" ref="getSomeStuffListenerAdapter" />
    <property name="concurrency" value="5-10" />
    <property name="transactionManager" ref="txManager" />
    <property name="sessionTransacted" value="true" /> 
</bean>

<bean id="atomikosTransactionManager"
      class="com.atomikos.icatch.jta.UserTransactionManager"
      depends-on="dataSource, atomikosConnectionFactory"
      init-method="init" destroy-method="close">
    <property name="forceShutdown" value="false" /> 
</bean>

<bean id="txManager"
      class="org.springframework.transaction.jta.JtaTransactionManager"> 
    <property name="transactionManager" ref="atomikosTransactionManager" />
</bean>

Ok we had this issue when running 3 Tomcat web applications on the same machine, using Atomikos, that were pointing to the same ActiveMQ server.

The applications were running in separate Tomcat instances with separate Atomikos log files.

The issue is that the XA XID (transaction IDs) are not unique across the three applications and this was causing ActiveMQ to get confused. This was due to the instances running on the same machine.

If you set Atomikos and ActiveMQ logging to TRACE. Then you can see that the same XID is generated by two or more applications.

This is because of the Atomikos properties setting: com.atomikos.icatch.tm_unique_name

If this is not set then the IP address of the machine is used when generating the XID, which was the same for the three applications and caused conflicts.

This issue was only likely to occur when the 3 Tomcat applications were started very close to each other.

So set the com.atomikos.icatch.tm_unique_name to be unique for each of the web applications and the problems went away.