<p>我必须安排一些看起来很复杂的任务并行运行。它们不依赖于彼此的结果,函数需要3个参数。在</p>
<p>我已经试过使用链式,地图和星图方法。有了链条,我得到了这个错误:</p>
<pre><code>[2019-04-23 15:28:00,991: ERROR/PoolWorker-3] Task proj.apps.tasks.generate[112a7426-5ac3-4cd6-8416-5591c3c018a3] raised unexpected: TypeError('get expected at least 1 arguments, got 0',)
Traceback (most recent call last):
File ".../local/lib/python2.7/site-packages/celery/app/trace.py", line 367, in trace_task
R = retval = fun(*args, **kwargs)
File ".../local/lib/python2.7/site-packages/celery/app/trace.py", line 622, in __protected_call__
return self.run(*args, **kwargs)
File ".../tasks.py", line 966, in generate
return res.get()
TypeError: get expected at least 1 arguments, got 0
</code></pre>
<p>使用<code>map</code>我不能传递所有参数,而使用<code>starmap</code>可以同时启动所有任务。在</p>
^{pr2}$
<p>任务示例:</p>
<pre class="lang-py prettyprint-override"><code>@shared_task
def generate(field1, field2, field3=None):
if field3 is not None:
return field1 + field2 + field3
return field1 + field2
</code></pre>
<p>使用链的代码:</p>
<pre><code>res = chain(generate.s(i, 5, j) for i in array1 for j in array2)
return res.get()
</code></pre>
<p>使用星图编码:</p>
<pre><code>arguments = [(i, 4, j) for i in array1 for j in array2]
~generate.starmap(arguments)
</code></pre>
<p>如果任务是真正独立的,则应该使用<code>.si</code>,而不是{<cd2>}:</p>
<pre><code>tasks = chain(generate.si(i, 5, j) for i in array1 for j in array2)
res = tasks()
return res.get()
</code></pre>