How to get full stack of StackOverflowError

Question

When observing a StackOverflowError how to retrieve the full call stack?

Consider this simple example:

public class Overflow {

    public Overflow() {
        new Overflow();
    }
    public static void a() {
        new Overflow();
    }
    public static void main(String[] argv) {
        a();
    }
}

Now the error reported is:

Exception in thread "main" java.lang.StackOverflowError
    at Overflow.<init>(Overflow.java:11)
    [last line repeated many times]

But I can't see the main and a method in the stack trace. My guess is this is because of overflow, the newest entry on the stack replaces the oldest one (?).

Now, how to get the a and main stack entries in the output?

The background is I get the a StackOverflowError (but that's not an infinite recursion, because it doesn't happen when increasing stack size) and it's hard to spot the problem in the code. I only get the multiple lines from java.util.regex.Pattern but not the information what code called that. The application is too complicated to set a breakpoint on each call to Patterns.

Answer 1

If you are running out of stack consider creating a dedicate thread w/ enough stack especially for running the request. Sample code below.

package t1;

import java.util.concurrent.Callable;
import java.util.concurrent.CancellationException;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.RejectedExecutionHandler;
import java.util.concurrent.SynchronousQueue;
import java.util.concurrent.ThreadFactory;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicLong;
import java.util.regex.Pattern;

public class RegExpRunner {
    ExecutorService svc;    
    public RegExpRunner(long stackSize){
        init(stackSize);

    }


    void init(long stackSize){
        final SynchronousQueue<Runnable> queue = new SynchronousQueue<Runnable>();

        svc = new ThreadPoolExecutor(1, 2, 60, TimeUnit.SECONDS,  queue, createThreadFactory(stackSize), new RejectedExecutionHandler(){//wait if there is a concurrent compile and no available threads
            @Override
            public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
                try{
                    queue.put(r);
                }catch(InterruptedException _ie){
                    Thread.currentThread().interrupt();
                    throw new IllegalStateException(_ie);
                }
            }                   
        });
    }

    private ThreadFactory createThreadFactory(final long stackSize) {       
        return new ThreadFactory(){
            final ThreadGroup g = Thread.currentThread().getThreadGroup();
            private final AtomicLong counter= new AtomicLong();
            {
                //take care of contextClassLoader and AccessControlContext              
            }

            @Override
            public Thread newThread(Runnable r) {               
                Thread t = new Thread(g, r, composeName(r), stackSize);
                return t;
            }

            protected String composeName(Runnable r) {
                return String.format("Regexp dedicated compiler: %d @ %tF %<tT ", counter.incrementAndGet(), System.currentTimeMillis());
            }   
        };
    };

    public Pattern compile(final String regex){//add flags if you need 'em
        Callable<Pattern> c = new Callable<Pattern>(){
            @Override
            public Pattern call() throws Exception {
                return Pattern.compile(regex);
            }           
        };

        try{
            Pattern p = svc.submit(c).get();
            return p;
        }catch(InterruptedException _ie){
            Thread.currentThread().interrupt();
            throw new IllegalStateException(_ie);
        } catch(CancellationException _cancel){
            throw new AssertionError(_cancel);//shan't happen
        } catch(ExecutionException _exec){
            Throwable t = _exec.getCause();
            if (t instanceof RuntimeException) throw (RuntimeException) t;
            if (t instanceof Error) throw (Error) t;
            throw new IllegalStateException(t==null?_exec:t);
        }


    }
}

Answer 2

As far as I know, it's not possible to get the full stack trace (however, I don't really know why).

However, what you could do to track down the problem, is to manually check for the stack depth in your affected code like this:

StackTraceElement[] trace = Thread.currentThread().getStackTrace();
if (trace.length > SOME_VALUE) {
  // trigger some diagnostic action, print a stack trace or have a breakpoint here
}

SOME_VALUE would need to be found by experimentation (high enough to not be triggered in "good" situations and low enough to not be unreachable). Of course this would slow down your code and should only be used for debugging the problem.

Update: I seem to have missed that the problem occurs in Pattern, which complicates matters. However, you could use a conditional method breakpoint at one of the Pattern methods in the stack trace with a condition like this (the actual value might need tweaking):

Thread.currentThread().getStackTrace().length > 300

This way you can find your own code at the bottom of the stack when you hit the breakpoint.

Answer 3

I would trigger a manual thread dump while I reproduce the issue. Most probably the stackoverflow is thrown only after some time. So, we can quickly trigger a Thread dump on jvm which will give us details about the caller by printing the entire stack of the problematic thread before its stack gets over flown.

Answer 4

I'd try plugging in something to decorate the stack trace output similar to ExceptionUtils to group repeated calls to the same Class or Package.

Answer 5

The JVM has an artificial limit of 1024 entries that you can have in the stack trace of an Exception or Error, probably to save memory when it occurs (since the VM has to allocate memory to store the stack trace).

Fortunately, there is a flag that allows to increase this limit. Just run your program with the following argument:

-XX:MaxJavaStackTraceDepth=1000000

This will print up to 1 million entries of your stack trace, which should be more than enough. It is also possible to set this value at 0 to set the number of entries as unlimited.

This list of non-standard JVM options gives more details:

Max. no. of lines in the stack trace for Java exceptions (0 means all). With Java > 1.6, value 0 really means 0. value -1 or any negative number must be specified to print all the stack (tested with 1.6.0_22, 1.7.0 on Windows). With Java <= 1.5, value 0 means everything, JVM chokes on negative number (tested with 1.5.0_22 on Windows).

Running the sample of the question with this flag gives the following result:

Exception in thread "main" java.lang.StackOverflowError
    at Overflow.<init>(Overflow.java:3)
    at Overflow.<init>(Overflow.java:4)
    at Overflow.<init>(Overflow.java:4)
    at Overflow.<init>(Overflow.java:4)
(more than ten thousand lines later:)
    at Overflow.<init>(Overflow.java:4)
    at Overflow.<init>(Overflow.java:4)
    at Overflow.a(Overflow.java:7)
    at Overflow.main(Overflow.java:10)

This way, you can find the original callers of the code that threw the Error, even if the actual stack trace is more than 1024 lines long.

If you can not use that option, there is still another way, if you are in a recursive function like this, and if you can modify it. If you add the following try-catch:

public Overflow() {
    try {
        new Overflow();
    }
    catch(StackOverflowError e) {
        StackTraceElement[] stackTrace = e.getStackTrace();
        // if the stack trace length is at  the limit , throw a new StackOverflowError, which will have one entry less in it.
        if (stackTrace.length == 1024) {
            throw new StackOverflowError();
        }
        throw e;  // if it is small enough, just rethrow it.
    }
}

Essentially, this will create and throw a new StackOverflowError, discarding the last entry because each one will be sent one level up compared to the previous one (this can take a few seconds, because all these Errors have to be created). When the stack trace will be reduced to 1023 elements, it is simply rethrown.

Ultimately this will print the 1023 lines at the bottom of the stack trace, which is not the complete stack trace, but is probably the most useful part of it.