More on python ImportError No module named

Question

Following the suggestion here, my package (or the directory containing my modules) is located at C:/Python34/Lib/site-packages. The directory contains an __init__.py and sys.path contains a path to the directory as shown.

Still I am getting the following error:

Traceback (most recent call last):
  File "C:/Python34/Lib/site-packages/toolkit/window.py", line 6, in <module>
    from catalogmaker import Catalog
  File "C:\Python34\Lib\site-packages\toolkit\catalogmaker.py", line 1, in <module>
    from patronmaker import Patron
  File "C:\Python34\Lib\site-packages\toolkit\patronmaker.py", line 4, in <module>
    class Patron:
  File "C:\Python34\Lib\site-packages\toolkit\patronmaker.py", line 11, in Patron
    patrons = pickle.load(f)
ImportError: No module named 'Patron'

I have a class in patronmaker.py named 'Patron' but no module named Patron so I am not sure what the last statement in the error message means. I very much appreciate your thoughts on what I am missing.

Python Version 3.4.1 on a Windows 32 bits machine.

Answer 1

You are saving all patron instances (i.e. self) to the Patron class attribute Patron.patrons. Then you are trying to pickle a class attribute from within the class. This can choke pickle, however I believe dill should be able to handle it. Is it really necessary to save all the class instances to a list in Patrons? It's a bit of an odd thing to do…

pickle serializes classes by reference, and doesn't play well with __main__ for many objects. In dill, you don't have to serialize classes by reference, and it can handle issues with __main__, much better. Get dill here: https://github.com/uqfoundation

Edit: I tried your code (with one minor change) and it worked.

dude@hilbert>$ python patronmaker.py

Then start python…

>>> import dill
>>> f = open('patrons.pkl', 'rb')
>>> p = dill.load(f)
>>> p
[Julius Caeser, Kunte Kinta, Norton Henrich, Mother Teresa]

The only change I made was to uncomment the lines at the end of patronmaker.py so that it saved some patrons…. and I also replaced import pickle with import dill as pickle everywhere.

So, even by downloading and running your code, I can't produce an error with dill. I'm using the latest dill from github.

Additional Edit: Your traceback above is from an ImportError. Did you install your module? If you didn't use setup.py to install it, or if you don't have your module on your PYTHONPATH, then you won't find your module regardless of how you are serializing things.

Even more edits: Looking at your code, you should be using the singleton pattern for patrons… it should not be inside the class Patron. The block of code at the class level to load the patrons into Patron.patrons is sure to cause problems… and probably bound to be the source of some form of errors. I also see that you are pickling the attribute Patrons.patrons (not even the class itself) from inside the Patrons class -- this is madness -- don't do it. Also notice that when you are trying to obtain the patrons, you use Patron.patrons… this is calling the class object and not an instance. Move patrons outside of the class, and use the singleton directly as a list of patrons. Also you should typically be using the patrons instance, so if you wanted to have each patron know who all the other patrons are, p = Patron('Joe', 'Blow'), then p.patrons to get all patrons… but you'd need to write a Patrons.load method that reads the singleton list of patrons… you could also use a property to make the load give you something that looks like an attribute.

If you build a singleton of patrons (as a list)… or a "registry" of patrons (as a dict) if you like, then just check if a patrons pickle file exists… to load to the registry… and don't do it from inside the Patrons class… things should go much better. Your code currently is trying to load a class instance on a class definition while it builds that class object. That's bad...

Also, don't expect people to go downloading your code and debugging it for you, when you don't present a minimal test case or sufficient info for how the traceback was created. You may have hit on a valid pickling error in dill for some dark corner case, but I can't tell b/c I can't reproduce your error. However, I can tell that you need some refactoring.

And just to be explicit:

Move your patrons initializing mess from Patrons into a new file patrons.py

import os
import dill as pickle

#Initialize patrons with saved pickle data
if os.path.isfile('patrons.pkl'):
    with open("patrons.pkl", 'rb') as f:
        patrons = pickle.load(f)
else: patrons = []

Then in patronmaker.py, and everywhere else you need the singleton…

import dill as pickle
import os.path
import patrons as the

class Patron:

    def __init__(self, lname, fname):
        self.lname = lname.title()
        self.fname = fname.title()
        self.terrCheckedOutHistory = {}
        #Add any created Patron to patrons list
        the.patrons.append(self)
        #Preserve this person via pickle
        with open('patrons.pkl', 'wb') as f:
            pickle.dump(the.patrons, f)

And you should be fine unless your code is hitting one of the cases that attributes on modules can't be serialized because they were added dynamically (see https://github.com/uqfoundation/dill/pull/47), which should definitely make pickle fail, and in some cases dill too… probably with an AtrributeError on the module. I just can't reproduce this… and I'm done.