As an additional question stemming from my previous question it turns out that pickle behaves differently in django shell compared to python shell...
this script:
import pickle
class TestObj(object): pass
testobj = TestObj()
pickled = pickle.dumps(testobj, pickle.HIGHEST_PROTOCOL)
will work fine in python shell, but in django shell will raise a PickleError along the lines of PicklingError: Can't pickle <class 'TestObj'>: attribute lookup __builtin__.TestObj failed
Is anyone able to explain the issue here? and if possible, link it back to my previous question?
pickle will make sure it can re-import a class, as only the data on the instance itself is pickled, plus the import location of the class. As such, pickle looks for the __module__ attribute on the class to determine where it came from.
It appears the Django interactive environment does not set this __module__ attribute; as a result TestObj.__module__ is inherited from the object base class instead, and that's __builtin__. Perhaps no __name__ global is set. As a result, the pickle module ends up looking in the wrong place for your class. There is no __builtin__.TestObj after all.
From the comments, I gather that you are trying to store mocked objects in the Django cache. That won't work, as mock objects are not pickleable. That makes sense, as on unpickling (which could be in an entirely new Python process), how would pickle know what original class was being mocked?
Could you not just use a serializer that doesn't pickle classes by reference… and then pre-serialize what you pass into the django cache? This would store not only the instance, but also the class definition itself -- which should allow you to reconstruct the instance anywhere.
See my answer on your original question:
Pickle can't store an object in django locmem cache during tests?
I don't know anything about django Mock objects, or if there's something especially unpicklable about them… But as long as they are built from python code (as opposed to built in C, and have a thin python wrapper layer), then the above should likely work.
