Good example of BadItem in Dill Module

Question

I'm exploring the detect method of Dill and am looking for a good simple example of a bad item - one that's unpicklable by Dill.

I first thought of a process and tried:

>>> proc = os.popen('ls -l')
>>> proc
<open file 'ls -l', mode 'r' at 0x10071d780>
>>> dill.detect.baditems(proc)
[]
>>> dill.dumps(proc)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/mikekilmer/Envs/env1/lib/python2.7/site-packages/dill/dill.py", line 143, in dumps
    dump(obj, file, protocol, byref)
  File "/Users/mikekilmer/Envs/env1/lib/python2.7/site-packages/dill/dill.py", line 136, in dump
    pik.dump(obj)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 224, in dump
    self.save(obj)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/Users/mikekilmer/Envs/env1/lib/python2.7/site-packages/dill/dill.py", line 557, in save_file
    position = obj.tell()
IOError: [Errno 29] Illegal seek

Which I guess would be expected if Dill uses seek to detectbaditems as you cannot seek on a PIPE.

Then I thought, surely globals() has something to offer. It again offered the same IOerror until proc was removed, and then yielded:

>>> dill.detect.baditems(globals)
[<module 'pickle' from '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.pyc'>, <module 'os' from '/Users/mikekilmer/Envs/env1/lib/python2.7/os.pyc'>, <__main__.Child object at 0x100776090>]

What would be a good simple example of an item that dill.detect would return as a bad item?

Answer 1

dill.detect.baditems is supposed to check for "bad items" inside of an object (i.e. check what is inside of an object that doesn't pickle). Maybe, at the top level it should check if the object itself pickles… it doesn't currently, and that could be misleading.

Here I'll demonstrate an unpicklable item that baditems says there's nothing unpicklable on the inside, which is true. Then I'll show how baditems finds unpicklable items inside of globals, and correctly identifies what cannot be pickled.

>>> x = iter([1,2,3,4,5])
>>> x
<listiterator object at 0x10d743510>
>>> import dill
>>> # everything inside a listiterator is serializable
>>> dill.detect.baditems(x)
[]
>>> # however, not everything in globals is serializable
>>> dill.detect.baditems(globals())
[<module '__builtin__' (built-in)>, <listiterator object at 0x10d743510>]

Hopefully, this doesn't seem too counter-intuitive.