Way to access resource files in python

Question

What is the proper way to access resources in python programs.

Basically in many of my python modules I end up writing code like that:

  DIRNAME = os.path.split(__file__)[0]

  (...) 

  template_file = os.path.join(DIRNAME, "template.foo")

Which is OK but:

• It will break if I will start to use python zip packages
• It is boilerplate code

In Java I had a function that did exactly the same --- but worked both when code was lying in bunch of folders and when it was packaged in .jar file.

Is there such function in Python, or is there any other pattern that I might use.

Answer 1

You'll want to look at using either get_data in the stdlib or pkg_resources from setuptools/distribute. Which one you use probably depends on whether you're already using distribute to package your code as an egg.

Answer 2

I guess the zipimport standard python module could be an answer...

EDIT: well, not the use of the module directly, but using sys.path as shown in the example could be a good way:

• I have a zip file test.zip with one python module test and a file test.foo inside
• to test that for the zipped python module test can be aware of of test.foo, it contains this code:

c

import os
DIRNAME = os.path.dirname(__file__)
if os.path.exists(os.path.join(DIRNAME, 'test.foo')):
    print 'OK'
else:
    print 'KO'

Test looks ok:

>>> import sys
>>> sys.path.insert(0, r'D:\DATA\FP12210\My Documents\Outils\SVN\05_impl\2_tools\test.zip')
>>> import test
OK
>>> 

So a solution could be to loop in your zip file to retrieve all python modules, and add them in sys.path; this piece of code would be ideally the 1st one loaded by your application.

Answer 3

Trying to understand how we could combine the two aspect togather

1. Loading for resources in native filesystem
2. Packaged in zipped files

Reading through the quick tutorial on zipimport : http://www.doughellmann.com/PyMOTW/zipimport/

I see the following example:

import sys
sys.path.insert(0, 'zipimport_example.zip')
import os
import zipimport
importer = zipimport.zipimporter('zipimport_example.zip')
module = importer.load_module('example_package')
print module.__file__
print module.__loader__.get_data('example_package/README.txt')

I think that output of __file__ is "zipimport_example.zip/example_package/__init__.pyc"

Need to check how it looks from inside.

But then we could always do something like this:

if ".zip" in example_package.__file__:
    ... 
    load using get_data
else:
    load by building the correct file path

[Edit:] I have tried to work out the example a bit better.

If the the package gets imported as zipped file then, two things happen

1. __file__ contains ".zip" in it's path.
2. __loader__ is available in the name space

If these two conditions are met then within the package you could do:

print __loader__.get_data(os.path.join('package_name','README.txt'))

else the module was loaded normally and you can follow the regular approach to loading the file.