Insights on SystemError: com_backpatch: offset too

Question

In python, "SystemError: com_backpatch: offset too large" is thrown when executing the code generated by the following:

f = open("test.py", "w")
f.write("def fn():\n   a =1000\n")
for a in xrange(3000):
   if a == 0:
      f.write("   if a == "+str(a)+": \n      print  "+str(a)+"\n")
   else:
      f.write("   elif a == "+str(a)+": \n      print  "+str(a)+"\n")



f.close()

import test

It is clear that if the length statement goes beyond a certain length, it throws this error.

Can someone give more insight into this error?

Answer 1

Accourding to this: http://www.cgl.ucsf.edu/pipermail/chimera-dev/2007/000404.html

The Python bytecode compiler has a limitation of a maximum of a 16 bit offset in a jump instruction. This means that you don't want to have 64K worth of characters in a single conditional block of code

More details here: http://www.mail-archive.com/[e-mail]/msg72631.html

Answer 2

JFYI, this script worked for me in a Debian Testing host with a 32bits userspace and kernel and Python 2.5.4.

$ ls -ln
total 4
-rw-r--r-- 1 1000 1000 270 2009-12-23 02:53 gentest.py
$ python gentest.py 
$ ls -ln
total 216
-rw-r--r-- 1 1000 1000    270 2009-12-23 02:53 gentest.py
-rw-r--r-- 1 1000 1000 111799 2009-12-23 02:58 test.py
-rw-r--r-- 1 1000 1000  93299 2009-12-23 02:58 test.pyc
$ uname -srvmo
Linux 2.6.30-2-486 #1 Thu Dec 3 23:32:25 UTC 2009 i686 GNU/Linux
$ python --version
Python 2.5.4

Answer 3

Looks like you are hitting a python interpreter limit. It appears that the branch from the start of the if to the end is too far - probably because offset is limited to 16 bits. If you change the "elif" to "if" then the problem goes away.

You need to reduce the size of the "if/elif" chain.

HIH

...richie