Using absolute_import and handling relative module

Question

I really hope this is a simple case of me miss-understanding the complex Python2 import mechanisms. I have the following setup:

$> ls -ltr pypackage1 
total 3
-rw-r--r-- 1 pelson pelson   0 Aug 17 19:20 io.py
-rw-r--r-- 1 pelson pelson   0 Aug 17 19:20 __init__.py
-rw-r--r-- 1 pelson pelson  57 Aug 17 19:22 code.py
$> cat pypackage1/code.py 
from __future__ import absolute_import

import zipfile

i.e. I have nothing but a stub package with an empty __init__.py and io.py, and a 2 lines code.py file.

I can import pypackage1:

$> python -c "import pypackage1.code"

But I cannot run the code.py file:

$> python pypackage1/code.py
Traceback (most recent call last):
  File "pypackage1/code.py", line 3, in <module>
    import zipfile
  File "python2.7/zipfile.py", line 462, in <module>
    class ZipExtFile(io.BufferedIOBase):
AttributeError: 'module' object has no attribute 'BufferedIOBase'

Clearly the problem has to do with the zipfile module picking up my relative io module over the builtin io module, but I thought my from __future__ import absolute_import would have fixed that.

Thanks in advance for any help,

Answer 1

One solution would be to put from __future__ import absolute_import in the zipfile.py module. Although your module is using absolute import, the zipfile module is not.

Another option is to not run from your package directory. You probably shouldn't be running the interpreter from within the package directory.

Answer 2

That's the correct behaviour. If you want to fix the error simply do not run from inside the package.

When you run a script which is inside the package, python wont interpret that directory as a package, thus adding the working directory to the PYTHONPATH. That's why the io module imported by the zipfile module is your io module and not the one inside the standard library.

I'd recommend to create a simple launcher script outside your package (or in a bin/scripts folder), and launch that. This script can simply contain something like:

from pypackage1 import code

code.main()

---

An alternative to this is to tell the python interpreter that the file that you want to execute is part of a module. You can do this using the -m command line option. In your case you would have to do:

python -m pypackage1.code

Note that the argument of -m should be the module name, not the file name.

Answer 3

File structure:

test.py
mylib/__init__.py
mylib/__collections.py
mylib/collections.py
mylib/mymod.py

This solution allows for:

• test.py to invoke both __builtin__.collections and mylib.collections
• mymod.py to invoke the above, both when it runs as part of the library and when it is run standalone (e.g. for test code)

In test.py:

from collections import deque
from mylib.collections import mydict

In mylib/__init__.py:

from __future__ import absolute_import
from . import collections
from . import mymod

In mylib/__collections.py:

class MyDict (dict):
    pass

In mylib/collections.py:

from __collections import *

In mylib/mymod.py:

from __future__ import absolute_import
from collections import deque
try:
    # Module running as part of mylib
    from .collections import MyDict
except ValueError:
    # Module running independently
    from __collections import MyDict

The above works with Python >=2.5. Python 3 doesn't need the lines 'from __future__ import absolute_import'.