存储列表列表的多处理队列

Question

我试图在一个进程中（在下面名为proc的函数中）将一个列表列表排队，然后在我调用event.set()之后让进程自行终止。从打印输出来看，我的函数proc总是结束，但进程本身仍在继续。如果我将在调用put中排队的列表数量降低（batchperq变量）（或者每个嵌套列表的大小更小），就可以实现这一点。在

import multiprocessing as mp
import queue
import numpy as np
import time

def main():
    trainbatch_q = mp.Queue(10)

    batchperq = 50  
    event = mp.Event()

    tl1 = mp.Process(target=proc,
                            args=( trainbatch_q, 20, batchperq, event))
    tl1.start()
    time.sleep(3)
    event.set()
    tl1.join()
    print("Never printed..")    

def proc(batch_q, batch_size, batchperentry, the_event):
    nrow = 100000 
    i0 = 0
    to_q = []
    while i0 < nrow:
        rowend = min(i0 + batch_size,nrow)
        somerows = np.random.randint(0,5,(rowend-i0,2))
        to_q.append(somerows.tolist())  
        if len(to_q) == batchperentry:
            print("adding..", i0, len(to_q))
            while not the_event.is_set():
                try: 
                    batch_q.put(to_q, block=False)
                    to_q = []
                    break
                except queue.Full:
                    time.sleep(1)
        i0 += batch_size                    
    print("proc finishes")

当我中断键盘时，我得到下面的轨迹。。。它试图获得的“锁”是什么？跟排队有关吗？在

^{pr2}$

Answer 1

进程从不退出的原因是您从未告诉它退出。我在函数的末尾添加了一个return，您的进程现在似乎正确退出了。在

def proc(batch_q, batch_size, batchperentry, the_event):
    nrow = 100000
    i0 = 0
    to_q = []
    while i0 < nrow:
        rowend = min(i0 + batch_size,nrow)
        somerows = np.random.randint(0,5,(rowend-i0,2))
        to_q.append(somerows.tolist())  
        if len(to_q) == batchperentry:
            print("adding..", i0, len(to_q))
            while not the_event.is_set():
                try: 
                    batch_q.put(to_q, block=False)
                    to_q = []
                    break
                except queue.Full:
                    time.sleep(1)
        i0 += batch_size                    
    print("proc finishes")
    return            # Added this line, You can have it return whatever is most relevant to you.

这是我运行的完整程序，包括我为使它成功退出所做的更改。在

^{pr2}$

希望这有帮助。在