显示几个进度条并一次更新它们而不打印额外的行

Question

Python版本：3.6

操作系统：Windows 10

我想用几个参数并行执行一个函数。为此，我将multiprocessing对象Pool与map方法一起使用。由于每个函数调用都会持续几个小时，所以我希望使用tqdmprogressbars来跟踪我的进度。在

为此，我尝试了以下代码：

from datetime import date, timedelta
from multiprocessing.pool import Pool

from tqdm import tqdm


def some_func(date1):
    for i in range(1000000):
        pass


def load_data(date1, n_process):
    dLength = 100
    progressbar = tqdm(range(0, dLength), position=n_process)
    for i in range(dLength):
        some_func(date1)
        progressbar.update(i)

if __name__ == '__main__':

    d1 = date(2018, 5, 22)
    d2 = date(2018, 5, 26)

    delta = d2 - d1

    dates_range = []

    for i in range(delta.days + 1):
        date1 = d1 + timedelta(i)
        dates_range.append(date1)
    n_processes = 5

    with Pool(n_processes) as p:

        p.starmap(load_data, zip(dates_range, range(n_processes)))

问题是progressbars开始重复，不便于跟踪：

0%| | 0/101373[00:00

0%| | 0/101373[00:05

0%| | 1/101373[00:09<；128:35:56，4.57s/it] 0%| | 0/217102[00:01

0%| | 0/214016[00:00

0%| | 3/101373[00:11<；97:28:47，3.46s/it]

0%| | 0/214016[00:01

0%| | 6/101373[00:12<；72:00:37，2.56s/it] 0%| | 1/217102[00:03<；81:07:30，1.35s/it]

0%| | 1/214016[00:02<；66:16:01，1.11s/it] 0%| | 3/217102[00:04<；66:47:19，1.11s/it]

如何将此文本转换为smth，如：

0%| | 0/101373[00:00

0%| | 0/101373[00:05

0%| | 0/217102[00:00

0%| | 0/101373[00:09<；128:35:56，4.57s/it]

0%| | 0/217102[00:01

然后，在相同的行中：

0%| | 1/101373[00:00

0%| | 5/101373[00:05

0%| | 20/217102[00:00

0%| | 15/101373[00:09<；128:35:56，4.57s/it]

0%| | 30/217102[00:01

等等。。。？在

Answer 1

我和你也有同样的问题。我想出来显示每个过程的进度条，以及一个进度条，指示整个进度，并额外维护所有进度条。multiprocessing.Manager().Queue()用于进程之间的通信（创建、更新和关闭进度栏）。这是我的测试代码。

import multiprocessing
from time import sleep
from tqdm import tqdm
def maintain_pbar(queue, num_of_tasks):
    """
    {
        "id": taskid,
        "cmd": "create", "update", "terminate" or "finish"
        "data": int, parameters for cmd
        "message": string for updating the description of tqdm progress bar
    }
    """
    pbars = dict()
    overall_pbar = tqdm(total=num_of_tasks, ncols=80, desc="Total progress")
    while True:
        message = queue.get(True)
        if message["cmd"] == "create":
            pbars[str(message["id"])] = tqdm(total=message["data"],
                                             ncols=80,
                                             desc="Task #" + str(message["id"]) + message["message"])
        elif message["cmd"] == "update":
            pbars[str(message["id"])].update(message["data"])
        elif message["cmd"] == "terminate":
            pbars[str(message["id"])].close()
            overall_pbar.update()
        elif message["cmd"] == "finish":
            overall_pbar.close()
            break

def progresser(params):
    n = params[0]
    queue = params[1]
    interval = 0.01 / (n + 2)
    total = 5000
    queue.put({"id": n, "cmd": "create", "data": total, "message": ""})
    for _ in range(total):
        sleep(interval)
        queue.put({"id": n, "cmd":"update", "data": 1, "message":""})
    queue.put({"id": n, "cmd": "terminate", "data": 1, "message": ""})

if __name__ == '__main__':
    pool = multiprocessing.Pool(4)
    queue = multiprocessing.Manager().Queue()
    l = [(i, queue) for i in range(20)]
    pool.apply_async(maintain_pbar, (queue, len(l)))
    pool.map(progresser, l)
    queue.put({"cmd": "finish"})
    pool.close()
    pool.join()

希望这有帮助！