Sandbox against malicious code in a Java applicati

Question

In a simulation server environment where users are allowed to submit their own code to be run by the server, it would clearly be advantageous for any user-submitted code to be run in side a sandbox, not unlike Applets are within a browser. I wanted to be able to leverage the JVM itself, rather than adding another VM layer to isolate these submitted components.

This kind of limitation appears to be possible using the existing Java sandbox model, but is there a dynamic way to enable that for just the user-submitted parts of a running application?

Answer 1

Obviously such a scheme raises all sorts of security concerns. Java has a rigorous security framework, but it isn't trivial. The possibility of screwing it up and letting an unprivileged user access vital system components shouldn't be overlooked.

That warning aside, if you're taking user input in the form of source code, the first thing you need to do is compile it to Java bytecode. AFIAK, this cannot be done natively, so you'll need to make a system call to javac, and compile the source code to bytecode on disk. Here's a tutorial that can be used as a starting point for this. Edit: as I learned in the comments, you actually can compile Java code from source natively using javax.tools.JavaCompiler

Once you have JVM bytecode, you can load it into the JVM using a ClassLoader's defineClass function. To set a security context for this loaded class you will need to specify a ProtectionDomain. The minimal constructor for a ProtectionDomain requires both a CodeSource and a PermissionCollection. The PermissionCollection is the object of primary use to you here- you can use it to specify the exact permissions the loaded class has. These permissions should be ultimately enforced by the JVM's AccessController.

There's a lot of possible points of error here, and you should be extremely careful to completely understand everything before you implement anything.