Memory Leak Looping cfmodule inside cffunction

Question

Googlers if you've got a heap dump with a root of coldfusion.runtime.CFDummyComponent read on.

Update 2/22/2011

Marc Esher of MXUnit fame found the exact same bug in a different context. His solution involves a large loop over a query solved by going from query="name" to from="1" to="#name.recordcount#" index="row". Another approach that works is using <cfthread> inside the loop as such:

<cfloop ...>
    <cfset threadName = "thread" & createUuid()>
    <cfthread name="#threadName#">
        <!--- do stuff --->
    </cfthread>
    <cfthread action="join" name="#threadName#">
</cfloop>

This is very effective when you run into situations where you need to do things inside the loop like queries and <cfmodule> inside <cffunction> so that the memory consumed is only for that iteration.

Old Question

Hoping someone else can confirm or tell me what I'm doing wrong. I am able to consistently reproduce an OOM running by calling the file oom.cfm (shown below). Using jconsole I am able to see the request consumes memory and never releases it until complete. The issue appears to be calling <cfmodule> inside of <cffunction>, where if I comment out the <cfmodule> call things are garbage collected while the request is running.

ColdFusion version: 9,0,1,274733

JVM Arguments

java.home=C:/Program Files/Java/jdk1.6.0_18
java.args=-server  -Xms768m -Xmx768m -Dsun.io.useCanonCaches=false -XX:MaxPermSize=512m -XX:+UseParallelGC -Xbatch -Dcoldfusion.rootDir={application.home}/ -Djava.security.policy={application.home}/servers/41ep8/cfusion.ear/cfusion.war/WEB-INF/cfusion/lib/coldfusion.policy -Djava.security.auth.policy={application.home}/servers/41ep8/cfusion.ear/cfusion.war/WEB-INF/cfusion/lib/neo_jaas.policy -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=56033

Test Case

oom.cfm (this calls template.cfm below - Adobe Bug #85736)

<cffunction name="fun" output="false" access="public" returntype="any" hint="">
    <cfset var local = structNew()/>
    <!--- comment out cfmodule and no OOM --->
    <cfmodule template="template.cfm">
</cffunction>

<cfset size = 1000 * 200>
<cfloop from="1" to="#size#" index="idx">
    <cfset fun()>
    <cfif NOT idx mod 1000>
        <cflog file="se-err" text="#idx# of #size#">
    </cfif>
</cfloop>

template.cfm

<!--- I am empty! --->

Update #2 (cfthread case from Elliott Sprehn - Adobe ColdFusion Bug #83359)

<cfthread name="test">  
  <cfloop from="1" to="10000" index="i">      
    <cflog text="This is very bad.">      
    <cflock name="test" timeout="10">      
    </cflock>  
  </cfloop>  
  <!--- Sleep a very long time (10 minutes) --->  
  <cfset sleep(600000)>
</cfthread>

Answer 1

I've not run into this before, but here's what I think is going on:

Each time cfmodule is called, a new memory space is created for it (which, IIRC, is the main difference between it and cfinclude). Because you are calling the cfmodule within the function, the cfmodule memory space technically belongs to that function's memory space. The function's memory is protected from garbage collection until the function is done. Result: heap fills, and you get an OOM error.

I don't think calling this a memory leak is correct, as it is behaving correctly, and when the function completes, the garbage collector can clear the hold on that memory. However, I can see how it might be inconvenient.

Answer 2

Here is a discussion of a possibly related Coldfusion version 9 cfc memory leak problem: http://forums.adobe.com/thread/1034324?start=0&tstart=0

See this bug report on it: https://bugbase.adobe.com/index.cfm?event=bug&id=3124148

I don't believe Adobe released a fix for verion 9.01 but supposedly this problem is fixed in version 10. There are workarounds for most people (depending on the scope of their problem) for this not unlike what's been described here.

Answer 3

This problem manifests with lots of tags unfortunately. I've seen this with cflock inside cfthread. Write a very long running loop in a cfthread that uses cflock, you'll run out of memory eventually. It takes a long time, but it happens. I bet the retention problem exists in regular requests too, but you don't usually have a loop that runs hundreds of thousands of times with a cflock inside so no one notices.

I reported this bug a long time ago, but it never got fixed: http://www.elliottsprehn.com/cfbugs/bugs/83359

The best solution for now is to not use cfmodule inside a loop like this. Custom tags really weren't intended for calling 20k times in a single request. You're going to want to use UDFs instead. cfmodule is extremely expensive anyway and using a UDF will be noticeably faster.