在windows上实现python多处理的最佳方法

2024-09-27 00:19:28 发布

您现在位置:Python中文网/ 问答频道 /正文

我必须做一个程序,读取一个PDF,转换成PNG文件的每一页,然后运行一些并行代码的每一个图像的每一页。我正在寻找一种方法来创建一个父进程,其中子进程的数目是恒定的。你知道吗

父亲必须给孩子们送工作。如果父亲有21页要处理,并且只有5个孩子,那么父亲必须管理一个队列来发送工作,而不杀死5个孩子并创建新的孩子。当孩子完成工作后,他给父亲发了一封信,让他做新的工作。你知道吗

我不想杀死子进程,因为我认为这比杀死并创建新的子进程或子进程要快。我走错方向了?你知道吗

我试着用多处理.apply\u async,但我没有找到我需要的方法。你知道吗

一些建议或指导?你知道吗

对不起,我的英语不好

我要做的是:

from multiprocessing  import Pool
import time
import random

def BarcodeSearcher(x):
    #Here goes the image processing    
    return x*x 

def resultCollector(result):
    print result

def main():
    pool = Pool(processes=3)
    for pag in range(3):
        pool.apply_async(BarcodeSearcher, args = (pag, ), callback = resultCollector) 
    pool.close()
    pool.join()

if __name__ == '__main__':
    main()

Tags: 方法importasync进程maindef孩子result
1条回答
网友
1楼 · 发布于 2024-09-27 00:19:28

你现在的工作方式应该很好。multiprocessing.Pool创建一个具有固定数量工作进程的池,所有工作进程都将在Pool的生存期内保持活动状态。Pool有一个内部队列,用于在其中一个工作进程完成工作后立即将工作项发送给工作进程。因此,您所需要做的就是将所有您想要完成的工作输入到Pool,然后Pool将处理所有这些工作进程的分发。你知道吗

考虑一下你的例子,除了现在我们给它30个工作项:

from multiprocessing import Pool, current_process
import time
import random

def BarcodeSearcher(x):
    print ("Process %s: handling %s" % (current_process(), x)
    #Here goes the image processing    
    return x*x 

def resultCollector(result):
    print result

def main():
    pool = Pool(processes=3)
    for pag in range(30):
        pool.apply_async(BarcodeSearcher, args = (pag, ), callback = resultCollector) 
    pool.close()
    pool.join()

if __name__ == '__main__':
    main()

输出如下:

Process <Process(PoolWorker-1, started daemon)>: handling 0
Process <Process(PoolWorker-3, started daemon)>: handling 2
Process <Process(PoolWorker-1, started daemon)>: handling 3
Process <Process(PoolWorker-1, started daemon)>: handling 4
Process <Process(PoolWorker-3, started daemon)>: handling 5
0
Process <Process(PoolWorker-2, started daemon)>: handling 1
9
4
Process <Process(PoolWorker-1, started daemon)>: handling 6
Process <Process(PoolWorker-1, started daemon)>: handling 7
16
Process <Process(PoolWorker-2, started daemon)>: handling 8
25
1
Process <Process(PoolWorker-3, started daemon)>: handling 9
36
49
64
81
Process <Process(PoolWorker-1, started daemon)>: handling 10
100
Process <Process(PoolWorker-2, started daemon)>: handling 11
Process <Process(PoolWorker-3, started daemon)>: handling 12
121
144
Process <Process(PoolWorker-2, started daemon)>: handling 13
Process <Process(PoolWorker-1, started daemon)>: handling 14
169
Process <Process(PoolWorker-2, started daemon)>: handling 15
196
Process <Process(PoolWorker-1, started daemon)>: handling 16
225
Process <Process(PoolWorker-3, started daemon)>: handling 17
256
Process <Process(PoolWorker-3, started daemon)>: handling 18
Process <Process(PoolWorker-1, started daemon)>: handling 19
Process <Process(PoolWorker-1, started daemon)>: handling 20
289
Process <Process(PoolWorker-1, started daemon)>: handling 21
324
Process <Process(PoolWorker-3, started daemon)>: handling 22
361
Process <Process(PoolWorker-3, started daemon)>: handling 24
400
Process <Process(PoolWorker-1, started daemon)>: handling 25
441
Process <Process(PoolWorker-3, started daemon)>: handling 26
Process <Process(PoolWorker-1, started daemon)>: handling 27
484
576
Process <Process(PoolWorker-3, started daemon)>: handling 28
Process <Process(PoolWorker-1, started daemon)>: handling 29
625
676
729
784
841
Process <Process(PoolWorker-2, started daemon)>: handling 23
529

如你所见,工作在你的工人之间分配,而你不必做任何特殊的事情。你知道吗

相关问题 更多 >

    热门问题