Distributing my Python scripts as JAR files with J

Question

I have been a Python programmer for almost two years, and I am used to writing small scripts to automate some repetitive tasks I had to do at the office. Now, apparently my colleagues noticed this, and they want those scripts too.

Some of them have Macs, some Windows; I made these on windows. I investigated the possibility of using py2exe or even py2app to make natives of my script, but they never satisfied me...

I came to know that all of them have JVM on their systems, so can I give them one single executable JAR file of my script using something like Jython may be?

How feasible is this... I mean, I had no idea how to write scripts for Jython, neither did I care about it when I wrote them... what kind of problems will it give?

Answer 1

The 'jythonc' command should be able to compile your .py source into JVM bytecode, which should make it portable to any Java install. Or so I read at: http://hell.org.ua/Docs/oreilly/other2/python/0596001886_pythonian-chp-25-sect-3.html

Answer 2

I experienced a similar issue in that I want to be able to create simple command line calls for my jython apps, not require that the user go through the jython installation process, and be able to have the jython scripts append library dependencies at runtime to sys.path so as to include core java code.

# append Java library elements to path
sys.path.append(os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "..", "lib", "poi-3.8-20120326.jar"))

When running the 'jython' launcher explicitly on the command line, on Unix systems, it just runs a big shell script to properly form a java command line call. This jython launcher seems to have a dependency on reaching back to a core install of jython, and by some way of magic allows the proper handling of .jar files being added to the sys.path at runtime from within my .py scripts. You can see what the call is and block execution by the following:

jython --print run_form.py
java -Xmx512m -Xss1024k -Dfile.encoding=UTF-8 -classpath /Applications/jython2.5.2/jython.jar: -Dpython.home=/Applications/jython2.5.2 -Dpython.executable=/Applications/jython2.5.2/bin/jython org.python.util.jython run_form.py

But it's still just firing up a JVM and running a class file. So my goal was to be able to make this java call to a standalone jython.jar present in my distribution's lib directory so users would not need to do any additional installation steps to start using my .py scripted utilities.

java -Xmx512m -Xss1024k -classpath ../../lib/jython.jar org.python.util.jython run_form.py

Trouble is that the behavior is enough different that I would get responses like this:

  File "run_form.py", line 14, in <module>
    import xls_mgr
  File "/Users/test/Eclipse/workspace/test_code/py/test/xls_mgr.py", line 17, in <module>
    import org.apache.poi.hssf.extractor as xls_extractor
ImportError: No module named apache

Now you might say that I should just add the jar files to the -classpath, which in fact I tried, but I would get the same result.

The suggestion of bundling all of your .class files in a jython.jar did not sound appealing to me at all. It would be a mess and would bind the Java/Python hybrid application too tightly to the jython distribution. So that idea was not going to fly. Finally, after lots of searching, I ran across bug #1776 at jython.org, which has been listed as critical for a year and a half, but I don't see that the latest updates to jython incorporate a fix. Still, if you're having problems with having jython include your separate jar files, you should read this.

http://bugs.jython.org/issue1776

In there, you will find the temporary workaround for this. In my case, I took the Apache POI jar file and unjar'ed it into its own separate lib directory and then modified the sys.path entry to point to the directory instead of the jar:

sys.path.append('/Users/test/Eclipse/workspace/test_code/lib/poi_lib')

Now, when I run jython by way of java, referencing my local jython.jar, the utility runs just peachy. Now I can create simple scripts or batch files to make a seamless command line experience for my .py utilities, which the user can run without any additional installation steps.

Answer 3

For distributing your Python scripts in a way that doesn't require a native Python installation, you could also try Nuitka, which basically translates your Python code to C++ code, which is then compiled to a true native binary.

Answer 4

The best current techniques for distributing your Python files in a jar are detailed in this article on Jython's wiki: http://wiki.python.org/jython/JythonFaq/DistributingJythonScripts

For your case, I think you would want to take the jython.jar file that you get when you install Jython and zip the Jython Lib directory into it, then zip your .py files in, and then add a __run__.py file with your startup logic (this file is treated specially by Jython and will be the file executed when you call the jar with "java -jar").

This process is definitely more complicated then in ought to be, and so we (the Jython developers) need to come up with a nice tool that will automate these tasks, but for now these are the best methods. Below I'm copying the recipe at the bottom of the above article (modified slightly to fit your problem description) to give you a sense of the solution.

Create the basic jar:

$ cd $JYTHON_HOME
$ cp jython.jar jythonlib.jar
$ zip -r jythonlib.jar Lib

Add other modules to the jar:

$ cd $MY_APP_DIRECTORY
$ cp $JYTHON_HOME/jythonlib.jar myapp.jar
$ zip myapp.jar Lib/showobjs.py
# Add path to additional jar file.
$ jar ufm myapp.jar othermanifest.mf

Add the __run__.py module:

# Copy or rename your start-up script, removing the "__name__  == '__main__'" check.
$ cp mymainscript.py __run__.py
# Add your start-up script (__run__.py) to the jar.
$ zip myapp.jar __run__.py
# Add path to main jar to the CLASSPATH environment variable.
$ export CLASSPATH=/path/to/my/app/myapp.jar:$CLASSPATH

On MS Windows, that last line, setting the CLASSPATH environment variable, would look something like this:

set CLASSPATH=C:\path\to\my\app\myapp.jar;%CLASSPATH%

Or, again on MS Windows, use the Control Panel and the System properties to set the CLASSPATH environment variable.

Run the application:

$ java -jar myapp.jar mymainscript.py arg1 arg2

Or, if you have added your start-up script to the jar, use one of the following:

$ java org.python.util.jython -jar myapp.jar arg1 arg2
$ java -cp myapp.jar org.python.util.jython -jar myapp.jar arg1 arg2
$ java -jar myapp.jar -jar myapp.jar arg1 arg2

The double -jar is kind of annoying, so if you want to avoid that and get the more pleasing:

$ java -jar myapp.jar arg1

You'll have to do a bit more work until we get something like this into a future Jython [Update: JarRunner is part of Jython 2.5.1]. Here is some Java code that looks for the __run__.py automatically, and runs it. Note that this is my first try at this class. Let me know if it needs improvement!

package org.python.util;

import org.python.core.imp;
import org.python.core.PySystemState;

public class JarRunner {

    public static void run(String[] args) {
        final String runner = "__run__";
        String[] argv = new String[args.length + 1];
        argv[0] = runner;
        System.arraycopy(args, 0, argv, 1, args.length);
        PySystemState.initialize(PySystemState.getBaseProperties(), null, argv);
        imp.load(runner);
    }

    public static void main(String[] args) {
        run(args);
    }
}

I put this code into the org.python.util package, since that's where it would go if we decide to include it in a future Jython. To compile it, you'll need to put jython.jar (or your myapp.jar) into the classpath like:

$ javac -classpath myapp.jar org/python/util/JarRunner.java

Then you'll need to add JarRunner.class to your jar (the class file will need to be in org/python/util/JarRunner.class) calling jar on the "org" directory will get the whole path into your jar.

$ jar uf org

Add this to a file that you will use to update the manifest, a good name is manifest.txt:

Main-Class: org.python.util.JarRunner

Then update the jar's manifest:

$ jar ufm myapp.jar manifest.txt

Now you should be able to run your app like this:

$ java -jar myapp.jar