<p>你的问题的解决办法是微不足道的。首先,请注意,<em>方法</em>不能被腌制。实际上,只有<a href="http://docs.python.org/2/library/pickle.html#what-can-be-pickled-and-unpickled" rel="nofollow">^{<cd1>}'s documentation</a>中列出的类型可以被pickle:</p>
<blockquote>
<ul>
<li><code>None</code>, <code>True</code>, and <code>False</code></li>
<li>integers, long integers, floating point numbers, complex numbers</li>
<li>normal and Unicode strings</li>
<li><code>tuple</code>s, <code>list</code>s, <code>set</code>s, and <code>dict</code>ionaries containing only picklable objects</li>
<li><em>functions <strong>defined at the top level of a module</strong></em></li>
<li><em>built-in functions <strong>defined at the top level of a module</strong></em></li>
<li><em>classes that are <strong>defined at the top level of a module</strong></em></li>
<li>instances of such classes whose <code>__dict__</code> or the result of calling <code>__getstate__()</code> is picklable (see section The pickle protocol
for details).</li>
</ul>
<p>[...]</p>
<p>Note that <em>functions</em> (built-in and user-defined) <em>are pickled by
“fully qualified” name reference, <strong>not by value</strong></em>. This means that
<strong>only the function name is pickled, along with the name of the module the function is defined in</strong>. Neither the function’s code, nor any of
its function attributes are pickled. Thus <em>the defining module must be
importable in the unpickling environment, and the module must contain
the named object</em>, otherwise an exception will be raised. [4]</p>
<p>Similarly, <strong>classes are pickled by named reference</strong>, so the same
restrictions in the unpickling environment apply. Note that <em>none of
the class’s code or data is pickled</em>[...]</p>
</blockquote>
<p>显然,方法不是在模块的顶层定义的函数,因此它不能被pickle(仔细阅读文档的这一部分,以避免pickle将来出现问题!)但是用一个全局函数替换该方法并将<code>self</code>作为附加参数传递是绝对简单的:</p>
<pre><code>import itertools as it
def global_fire(argument):
self, obj = argument
self.fire(obj)
class Collectorparallel():
def fire(self,obj):
collectorController = Collectorcontroller()
collectorController.crawlTask(obj)
def start(self):
log_to_stderr(logging.DEBUG)
pluginObjectList = []
for pluginName in self.settingModel.getAllCollectorName():
name = pluginName.capitalize()
#Get plugin class and instanitiate object
module = __import__('plugins.'+pluginName,fromlist=[name])
pluginClass = getattr(module,name)
pluginObject = pluginClass()
pluginObjectList.append(pluginObject)
pool = Pool(8)
jobs = pool.map(global_fire, zip(it.repeat(self), pluginObjectList))
pool.close()
print pluginObjectList
</code></pre>
<p>注意,由于<code>Pool.map</code>只使用一个参数调用给定函数,因此我们必须将<code>self</code>和实际参数“打包”在一起。为此,我有<code>zip</code>ped<code>it.repeat(self)</code>和原始的iterable。在</p>
<p>如果您不关心调用的完成顺序,那么使用<code>pool.imap_unordered</code><em>可能会提供更好的性能。但是它返回一个iterable而不是一个list,因此如果您想要结果列表,您必须执行<code>jobs = list(pool.imap_unordered(...))</code>。在</p>