StackOverflowError at URLFetchServiceFactory.getUR

Question

My application uses this API from GAE:

URLFetchServiceFactory.getURLFetchService().fetch

Which works running on devserver but after 30 to 60 minutes (time varies) it starts throwing StackOverflowError.

In what way that a call to fetch to a standard web URL could actually cause URLFetchServiceFactory.getURLFetchService().fetch to throw SO error.

Here's the complete log: https://pastebin.com/WCcmXCH3 (long)

Answer 1

I have found the culprit of the StackOverflowError after several hours of digging, apparently the for-loop before the call to fetch is causing this, here:

List<String> cachedLinks = (List<String>) memcache.get(url);
    for(String link : cachedLinks) { // size range from 0 to 1000 or more...
        // Do something
    }
// then a bit later
URLFetchServiceFactory.getURLFetchService().fetch(...);

Whereas, the cachedLinks contains thousands of String values. Removing the for-loop stopped the throwing of StackOverflowError.

Maybe a Java guru can better explain this phenomenon better and why the for-loop above the call to fetch call.

Answer 2

The stack depth in the complete log has a very suspicious value of 1025 = 1024 +1.

A closer look reveals that you have this repeating recursive call sequence:

[INFO]  at java.io.File.isDirectory(File.java:844)
[INFO]  at sun.net.www.ParseUtil.fileToEncodedURL(ParseUtil.java:269)
[INFO]  at sun.security.provider.PolicyFile.canonicalizeCodebase(PolicyFile.java:1735)
[INFO]  at sun.security.provider.PolicyFile.access$700(PolicyFile.java:258)
[INFO]  at sun.security.provider.PolicyFile$5.run(PolicyFile.java:1188)
[INFO]  at sun.security.provider.PolicyFile$5.run(PolicyFile.java:1186)
[INFO]  at java.security.AccessController.doPrivileged(Native Method)
[INFO]  at sun.security.provider.PolicyFile.getPermissions(PolicyFile.java:1185)
[INFO]  at sun.security.provider.PolicyFile.getPermissions(PolicyFile.java:1132)
[INFO]  at sun.security.provider.PolicyFile.implies(PolicyFile.java:1086)
[INFO]  at com.google.appengine.tools.development.IsolatedAppClassLoader$ProxyPolicy$2.run(IsolatedAppClassLoader.java:418)
[INFO]  at com.google.appengine.tools.development.IsolatedAppClassLoader$ProxyPolicy$2.run(IsolatedAppClassLoader.java:415)
[INFO]  at java.security.AccessController.doPrivileged(Native Method)
[INFO]  at com.google.appengine.tools.development.IsolatedAppClassLoader$ProxyPolicy._implies(IsolatedAppClassLoader.java:415)
[INFO]  at com.google.appengine.tools.development.IsolatedAppClassLoader$ProxyPolicy.implies(IsolatedAppClassLoader.java:42005)
[INFO]  at java.security.ProtectionDomain.implies(ProtectionDomain.java:285)
[INFO]  at java.security.AccessControlContext.checkPermission(AccessControlContext.java:450)
[INFO]  at java.security.AccessController.checkPermission(AccessController.java:884)
[INFO]  at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
[INFO]  at com.google.appengine.tools.development.DevAppServerFactory$CustomSecurityManager.checkPermission(DevAppServerFactory.java:429)
[INFO]  at java.lang.SecurityManager.checkRead(SecurityManager.java:888)
[INFO]  at java.io.File.isDirectory(File.java:844)
... repeating 49 times or so

I'm not a Java user, but if I interpret it correctly it appears you may be processing a directory structure too deep.

If so, either re-arrange the directory structure or modify your code to limit how deep it goes into it in one-shot.