Python:无法pickle模块对象

Question

我想上一堂课

TypeError: can't pickle module objects

尽管浏览了一下网络，我还是搞不清这到底是什么意思。我也不确定是哪个module object引起了麻烦。有办法找到罪犯吗？堆栈跟踪似乎没有任何指示。

Answer 1

我可以这样复制错误消息：

import cPickle

class Foo(object):
    def __init__(self):
        self.mod=cPickle

foo=Foo()
with file('/tmp/test.out', 'w') as f:
    cPickle.dump(foo, f) 

# TypeError: can't pickle module objects

是否有引用模块的类属性？

Answer 2

Python无法pickle模块对象是真正的问题。有什么好理由吗？我不这么认为。模块对象不可访问导致了python作为并行/异步语言的脆弱性。如果要pickle模块对象，或者python中的几乎任何对象，那么使用dill。

Python 3.2.5 (default, May 19 2013, 14:25:55) 
[GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import dill
>>> import os
>>> dill.dumps(os)
b'\x80\x03cdill.dill\n_import_module\nq\x00X\x02\x00\x00\x00osq\x01\x85q\x02Rq\x03.'
>>>
>>>
>>> # and for parlor tricks...
>>> class Foo(object):
...   x = 100
...   def __call__(self, f):
...     def bar(y):
...       return f(self.x) + y
...     return bar
... 
>>> @Foo()
... def do_thing(x):
...   return x
... 
>>> do_thing(3)
103 
>>> dill.loads(dill.dumps(do_thing))(3)
103
>>> 

在这里获取dill：https://github.com/uqfoundation/dill

Answer 3

根据文件：

What can be pickled and unpickled?

The following types can be pickled:

• None, True, and False

• integers, floating point numbers, complex numbers

• strings, bytes, bytearrays

• tuples, lists, sets, and dictionaries containing only picklable objects

• functions defined at the top level of a module (using def, not lambda)

• built-in functions defined at the top level of a module

• classes that are defined at the top level of a module

• instances of such classes whose __dict__ or the result of calling __getstate__() is picklable (see section Pickling Class Instances for details).

如您所见，模块不在列表中。注意，这在使用deepcopy时也是正确的，而且不仅适用于pickle模块，如deepcopy文档中所述：

This module does not copy types like module, method, stack trace, stack frame, file, socket, window, array, or any similar types. It does “copy” functions and classes (shallow and deeply), by returning the original object unchanged; this is compatible with the way these are treated by the pickle module.

可能的解决方法是使用@property装饰器而不是属性。 例如，这应该有效：

    import numpy as np
    import pickle

    class Foo():
        @property
        def module():
            return np

    foo = Foo()
    with open('test.out', 'wb') as f:
        pickle.dump(foo, f)