擅长:python、mysql、java
<p>您的代码有一个错误:</p>
<pre><code>for line in sys.stdin:
lines.append(line)
if len(lines) >= 100000:
pool.map_async(processLine, lines, 2000)
</code></pre>
<p>这要等到<code>lines</code>累积超过100000行。之后,<code>pool.map_async</code>被调用到100000多行的整个列表中,每个额外的行都被调用。</p>
<p>不清楚你到底想做什么,但是
如果不需要返回值,请使用<code>pool.apply_async</code>,而不是<code>pool.map_async</code>。可能是这样的:</p>
<pre><code>import multiprocessing as mp
def processLine(line):
#process something
print "result"
if __name__ == '__main__':
pool = mp.Pool(processes = 8)
for line in sys.stdin:
pool.apply_async(processLine, args = (line, ))
pool.close()
pool.join()
</code></pre>