python3多处理只使用一个内核，4个线程打开

Question

我正在写一个简单的素数计算器，我正在努力使它尽可能快。我一直在研究python中的multiprocessing，而且我所看到的每一个地方，似乎打开4个进程将启动4个线程。。。这是有道理的。但是，当运行以下代码时，即使有4个进程处于打开状态，我也只能得到1个进程（在top/Activity Monitor（Mac）/etc中）。我将primesPerThread设置为500，这样进程在完成之前将持续更长的时间，以便我可以查找它们（在top/etc中）。但是，我只看到一个Python进程使用大约100%的CPU。在multiprocessing中是否有我不理解/使用的东西？到目前为止，我得到的是：

import multiprocessing
import math
import os
import time

def mod(numerator, denominator):
    result = numerator % denominator
    return result

def checkPrimes(low, high):
    global primes
    reult = 0
    curPrime = low
    notPrime = False

    for i in range(low, high):
        notPrime = False
        for j in range(int(math.floor(math.sqrt(len(primes))))):
            result = mod(curPrime, primes[j])
            if result == 0:
                notPrime = True
                break
        if not notPrime:
            primes.append(curPrime)
        curPrime += 1

cores = 4
primes = [2,3]
curPrime = 4
result = 0
notPrime = False
primesPerThread = 500
numPrimes = 0
prevNumPrimes = 0

processes = []

while len(primes) < 1000000:
    for c in range(0, cores):
        low = curPrime
        high = curPrime + primesPerThread
        process = multiprocessing.Process(target=checkPrimes(low, high))
        process.start()
        processes.append(process)
        curPrime += primesPerThread
    for process in processes:
        process.join()
    prevNumPrimes = numPrimes
    numPrimes = len(primes)
    print(len(primes))
    # for i in range(numPrimes - prevNumPrimes, 0, -1):
    #     os.popen("echo " + str(primes[len(primes) - i]) + " >> primes.txt")

Answer 1

多处理不是线程！顾名思义，它使用不同的进程而不是线程。因此，使用全局变量是行不通的，因为它不是在进程之间共享的。在

使用列表理解通常是for循环的先决条件，因为它更快。考虑一下这一点

In [1]: import math

In [2]: %cpaste
Pasting code; enter ' ' alone on the line to stop or use Ctrl-D.
:def prime_list(num):
:    if num < 3:
:        raise ValueError('this function only accepts arguments > 2')
:    # For numbers >2, only odd numbers are prime.
:    candidates = range(3, num+1, 2) 
:    # For an odd number c to be a prime, one must ensure that 
:    # c modulo all previous odd numbers (p) must be non-zero.
:    L = [c for c in candidates if all(c % p != 0 for p in
:         range(3, int(math.sqrt(c))+1, 2))]
:    # 2 is also a prime number.
:    return [2] + L
: 

In [3]: %timeit prime_list(100000)
1 loops, best of 3: 610 ms per loop

In [4]: rv =  prime_list(100000)

In [5]: len(rv)
Out[5]: 9592

In [6]: rv[:10]
Out[6]: [2, 3, 5, 7, 11, 13, 17, 19, 23, 29]

In [7]: rv[-10:]
Out[7]: [99877, 99881, 99901, 99907, 99923, 99929, 99961, 99971, 99989, 99991]

这个算法只需0.6秒就可以找到前10万个数中的素数。当然，当你把要搜索的数字列得更长时，这个时间会迅速攀升；你必须检查所有奇数直到√c

^{pr2}$

您可以通过向prime_list添加要检查的最小数目，然后使用multiprocessing.Pool对象中的map方法将工作分散到多个实例上。在

import math
import multiprocessing

def prime_list2(arg):
    start, num = arg # Unpack the tuple.
    if num < 3:
        raise ValueError('this function only accepts arguments > 2')
    if start < 3:
       start = 3
    candidates = range(start, num+1, 2)
    L = [c for c in candidates if all(c % p != 0 for p in
         range(3, int(math.sqrt(c))+1, 2))]
    if start < 3:
        L = [2] + L
    return L

data = [(n*100000+1, (n+1)*100000+1) for n in range(0,10)]
p = multiprocessing.Pool()
rv = p.map(prime_list2, data)
print('len(rv)=', len(rv))
for i, v in enumerate(rv):
    print('rv[{}][:10] ='.format(i), v[:10])

这样可以得到：

len(rv)= 10
rv[0][:10] = [3, 5, 7, 11, 13, 17, 19, 23, 29, 31]
rv[1][:10] = [100003, 100019, 100043, 100049, 100057, 100069, 100103, 100109, 100129, 100151]
rv[2][:10] = [200003, 200009, 200017, 200023, 200029, 200033, 200041, 200063, 200087, 200117]
rv[3][:10] = [300007, 300017, 300023, 300043, 300073, 300089, 300109, 300119, 300137, 300149]
rv[4][:10] = [400009, 400031, 400033, 400051, 400067, 400069, 400087, 400093, 400109, 400123]
rv[5][:10] = [500009, 500029, 500041, 500057, 500069, 500083, 500107, 500111, 500113, 500119]
rv[6][:10] = [600011, 600043, 600053, 600071, 600073, 600091, 600101, 600109, 600167, 600169]
rv[7][:10] = [700001, 700027, 700057, 700067, 700079, 700081, 700087, 700099, 700103, 700109]
rv[8][:10] = [800011, 800029, 800053, 800057, 800077, 800083, 800089, 800113, 800117, 800119]
rv[9][:10] = [900001, 900007, 900019, 900037, 900061, 900089, 900091, 900103, 900121, 900139]