Unpickle sometimes makes blank objects

Question

I'm trying to use pickle to save a custom class; something very much like the code below (though with a few methods defined on the class, and several more dicts and such for data). However, often when I run this, pickle and then unpickle, I lose whatever data was in the class, and its as if I created a new blank instance.

import pickle
class MyClass:
    VERSION = 1
    some_data = {}
    more_data = set()

    def save(self,filename):
        with open(filename, 'wb') as f:
            p = pickle.Pickler(f)
            p.dump(self)

    def load(filename):
        with open(filename,'rb') as ifile:
            u = pickle.Unpickler(ifile)
            obj = u.load()
            return obj

I was wondering if this had something to do with the memo of the pickle class, but I don't feel like it should. When it doesn't work, I look at my generated file and it looks something like this: (Obviously not meant to be readable, but it obviously contains no data)

€c__main__
MyClass
q

Anyways, I hope this is enough for someone to understand what might possibly be going on here, or what to look at.

Answer 1

The problem you're having is that you're using mutable class variables to hold your data, rather than putting the data into instance variables.

The pickle module only saves the data stored directly on the instance, not class variables that can also be accessed via self. When you're finding your unpickled instance have no data, what that probably means is that the class doesn't hold the data from the previous run, so the instances can't access it any more.

Using class variables that way will probably cause you other problems too, as the data will be shared by all instances of the class! Here's a Python console session code that illustrates the issue:

>>> class Foo(object):
        class_var = []
        def __init__(self, value):
            self.class_var.append(value)

>>> f1 = Foo(1)
>>> f1.class_var
[1]
>>> f2 = Foo(2)
>>> f2.class_var
[1, 2]

That's probably not what you wanted. But it gets worse!

>>> f1.class_var
[1, 2] 

The data you thought had belonged to f1 has been changed by the creation of f2. In fact, f1.class_var is the very same object as f2.class_var (it is also available via Foo.class_var directly, without going through any instances at all).

So, using a class variable is almost certainly not what you want. Instead, write an __init__ method for the class that creates a new value and saves it as an instance variable:

>>> class Bar(object):
        def __init__(self, value):
            self.instance_var = [] # creates a separate list for each instance!
            self.instance_var.append(value)

>>> b1 = Bar(1)
>>> b1.instance_var
[1]
>>> b2 = Bar(2)
>>> b2.instance_var # doesn't include value from b1
[2]
>>> b1.instance_var # b1's data is unchanged
[1]

Pickle will handle this class as you expect. All of its data is in the instances, so you should never end up with an empty instance when you unpickle.