Strange JRUN issue. JRUN eating up 50% of memory f

Question

I have strange JRUN issue. I have installed ColdFusion 9.0 on Amazon EC2 instance and seems everything working good except JRUN eating up arround 50% of memory for particular timespan.

For countinous two hours it take 50% of CPU usage and then next 45 min to an hour it work normally and again it take 50% for next two hours. I am not running any schedule file.

Also I will appreciate if anyone guide me how we can know which request causing JRUN to eat memory.

Thanks for suggestion MIKE, but it seems that I already enable matrix but cann't find any changes when JRUN was running normal and taking arround 50% memory. As limitation of number of character I am posting log as separate answer. Also I have tried to stop IIS to make sure no external request come to ColdFusion and FusionReactor shows no requests to JRUN but still using high memoery.

Created by JRun on 09/22 08:00:35

09/22 08:00:35 metrics Web threads (busy/total): 1/31 Sessions: 0 Total Memory=684672 Free=228809 09/22 08:01:35 metrics Web threads (busy/total): 2/34 Sessions: 0 Total Memory=761792 Free=364733 09/22 08:02:35 metrics Web threads (busy/total): 1/34 Sessions: 0 Total Memory=773568 Free=338352 09/22 08:03:35 metrics Web threads (busy/total): 1/35 Sessions: 0 Total Memory=781696 Free=283261 09/22 08:04:35 metrics Web threads (busy/total): 3/36 Sessions: 0 Total Memory=790784 Free=325807 09/22 08:05:35 metrics Web threads (busy/total): 1/36 Sessions: 0 Total Memory=794432 Free=301484 09/22 08:06:35 metrics Web threads (busy/total): 1/35 Sessions: 0 Total Memory=768640 Free=221172 09/22 08:07:35 metrics Web threads (busy/total): 1/38 Sessions: 0 Total Memory=793984 Free=245422 09/22 08:08:35 metrics Web threads (busy/total): 1/37 Sessions: 0 Total Memory=790080 Free=376290 09/22 08:09:35 metrics Web threads (busy/total): 1/38 Sessions: 0 Total Memory=792832 Free=307553 09/22 08:10:35 metrics Web threads (busy/total): 1/36 Sessions: 0 Total Memory=792000 Free=337115 09/22 08:11:35 metrics Web threads (busy/total): 1/36 Sessions: 0 Total Memory=789184 Free=240118 09/22 08:12:35 metrics Web threads (busy/total): 1/37 Sessions: 0 Total Memory=789440 Free=342658 09/22 08:13:35 metrics Web threads (busy/total): 1/39 Sessions: 0 Total Memory=787520 Free=347211

After this stage JRUN was running at 50% CPU Usage.

09/22 08:14:35 metrics Web threads (busy/total): 1/39 Sessions: 0 Total Memory=770112 Free=211797 09/22 08:15:35 metrics Web threads (busy/total): 1/37 Sessions: 0 Total Memory=726208 Free=249031 09/22 08:16:35 metrics Web threads (busy/total): 1/38 Sessions: 0 Total Memory=715392 Free=158240 09/22 08:17:35 metrics Web threads (busy/total): 1/39 Sessions: 0 Total Memory=705600 Free=239585 09/22 08:18:35 metrics Web threads (busy/total): 1/36 Sessions: 0 Total Memory=718848 Free=175842 09/22 08:19:35 metrics Web threads (busy/total): 1/36 Sessions: 0 Total Memory=687488 Free=204397 09/22 08:20:35 metrics Web threads (busy/total): 1/36 Sessions: 0 Total Memory=701440 Free=185422 09/22 08:21:35 metrics Web threads (busy/total): 1/35 Sessions: 0 Total Memory=671744 Free=154754 09/22 08:22:35 metrics Web threads (busy/total): 2/35 Sessions: 0 Total Memory=664320 Free=163835 09/22 08:23:35 metrics Web threads (busy/total): 1/33 Sessions: 0 Total Memory=674752 Free=195576 09/22 08:24:35 metrics Web threads (busy/total): 1/35 Sessions: 0 Total Memory=661760 Free=203445 09/22 08:25:35 metrics Web threads (busy/total): 1/35 Sessions: 0 Total Memory=656576 Free=174511 09/22 08:26:35 metrics Web threads (busy/total): 1/35 Sessions: 0 Total Memory=651968 Free=194924 09/22 08:27:35 metrics Web threads (busy/total): 1/35 Sessions: 0 Total Memory=632896 Free=152896 09/22 08:28:35 metrics Web threads (busy/total): 1/36 Sessions: 0 Total Memory=633984 Free=215603 09/22 08:29:35 metrics Web threads (busy/total): 1/34 Sessions: 0 Total Memory=630720 Free=198136 09/22 08:30:35 metrics Web threads (busy/total): 2/35 Sessions: 0 Total Memory=616512 Free=140867 09/22 08:31:35 metrics Web threads (busy/total): 1/36 Sessions: 0 Total Memory=613824 Free=140683 09/22 08:32:35 metrics Web threads (busy/total): 1/36 Sessions: 0 Total Memory=605184 Free=166131 09/22 08:33:35 metrics Web threads (busy/total): 1/37 Sessions: 0 Total Memory=608448 Free=132906 09/22 08:34:35 metrics Web threads (busy/total): 1/37 Sessions: 0 Total Memory=609344 Free=180291 09/22 08:35:35 metrics Web threads (busy/total): 1/36 Sessions: 0 Total Memory=603008 Free=161821 09/22 08:36:35 metrics Web threads (busy/total): 2/36 Sessions: 0 Total Memory=604672 Free=150526 09/22 08:37:35 metrics Web threads (busy/total): 1/37 Sessions: 0 Total Memory=606144 Free=162952 09/22 08:38:35 metrics Web threads (busy/total): 1/36 Sessions: 0 Total Memory=602048 Free=136201 09/22 08:39:35 metrics Web threads (busy/total): 1/36 Sessions: 0 Total Memory=606656 Free=116793 09/22 08:40:35 metrics Web threads (busy/total): 1/37 Sessions: 0 Total Memory=602880 Free=120984 09/22 08:41:35 metrics Web threads (busy/total): 1/36 Sessions: 0 Total Memory=607424 Free=112235 09/22 08:42:35 metrics Web threads (busy/total): 1/35 Sessions: 0 Total Memory=607424 Free=135657

Answer 1

As Dan Short queries here it would be better if you get full detail about what JRun or more importantly, the JVM is doing. I have worked on these issues many times and suggest you enable "metrics and GC logging". You can find details of how to do this in these two blog posts...

http://www.cfwhisperer.com/post.cfm/10-steps-to-a-stable-and-performant-web-application-step-2

http://www.cfwhisperer.com/post.cfm/10-steps-to-a-stable-and-performant-web-application-step-3

Once you have this logging enabled we can actually see what is going on and I would suggest at least 24 hours of logging to be accurate.

Answer 2

After spending much time on it I figure it out that client variables stored in registry was causing whole issue and ColdFusion thread for purging client variable which was running every hour taking too much CPU usage. Here is full story.

http://www.thecfguy.com/post.cfm/strange-coldfusion-issue-jrun-eating-up-to-50-of-cpu