Pickling error: Can't pickle

问题:

I am wondering what this error might mean:

PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

I understand that it has something to do with using multiple cores. I am running my program on a cluster and using 15 threads in this line of my code:

gauss2 = PTSampler(ntemps, renwalkers, rendim, lnlike, lnprior, threads=15)

The sampler in question is the one documented for the Parallel Tempering sampler at http://dan.iel.fm/emcee/current/user/pt/

Any idea what this error might mean?

回答1:

The error means you are trying to pickle a builtin FunctionType… not the function itself. It's likely do to a coding error somewhere picking up the class of the function instead of the function itself.

>>> import sys
>>> import pickle
>>> import types
>>> types.FunctionType
<type 'function'>
>>> try:
...     pickle.dumps(types.FunctionType)
... except:
...     print sys.exc_info()[1]
... 
Can't pickle <type 'function'>: it's not found as __builtin__.function
>>> def foo(x):
...   return x
... 
>>> try:
...     pickle.dumps(type(foo))
... except:
...     print sys.exc_info()[1]
... 
Can't pickle <type 'function'>: it's not found as __builtin__.function
>>> try:
...     pickle.dumps(foo.__class__)
... except:
...     print sys.exc_info()[1]
... 
Can't pickle <type 'function'>: it's not found as __builtin__.function
>>> pickle.dumps(foo)
'c__main__\nfoo\np0\n.'
>>> pickle.dumps(foo, -1)
'\x80\x02c__main__\nfoo\nq\x00.'

If you have a FunctionType object, then all you need to do is get one of the instances of that class -- i.e. a function like foo.