多处理。池无限循环（并行处理不会停止）

Question

这是我的用户定义函数

yms = ['201901', '201902', '201903', '201904', '201905', .... , '201812']

def minc(ym):
    print('MINC %s %s\n' %(ym, str(datetime.datetime.now())))
    print('value %s is in PID : %s \n' %(ym, os.getpid()))
    t = datetime.datetime.now()
    minc1 = pd.read_sql("""
    select substring(MINC_IN_YM, 1, 4) as YEAR, substring(MINC_IN_YM, 5, 2) as MONTH,
    count(MINC_INSP_NO) as NROWS,
    sum(MINC_OKQTY) as TOTAL_QUANTITY,
    sum(MINC_AV_PRICE*MINC_OKQTY) as TOTAL_DOLLARS
    from dwadm.W_MINC
    where MINC_INC_INF in ('RN', 'CN')
    and MINC_ACCID in ('A', 'G', 'V')
    and MINC_IN_YM = '%s'
    and substring(MINC_BRNCD, 1, 1) not in ('S', 'C')
    GROUP BY YEAR, MONTH, MINC_BRNCD, MINC_BRNCD_WHS, MINC_VNDCD, MINC_PTNO
    """ % ym, conn)
    print('MINC ends %s1 %s\n' %(ym, str(datetime.datetime.now()), str(datetime.datetime.nows())
    
    return minc1

我想用函数“minc（）”和下面的函数“parallelized（）”进行多处理

def parallelized():
    if __name__ == '__main__' :
      pool = Pool(processes = 8)
      df = pool.map(minc, yms)
      pool.close()
      pool.join()

parallelized()

或

if __name__ == '__main__':
    freeze_support()
    t1 = time.time()
    pool = Pool(8)
    pool.map(minc, yms)
    pool.close()
    pool.join()

但当我运行上述代码时，这些代码不会停止。他们在无限地奔跑。。。 我不知道如何解决这个问题。 当我不使用用户定义的函数“minc（）”而是使用“sum（）”或任何其他基本函数时，我可以意识到并行处理执行得很好。 是否有其他解决方案可以很好地执行用户定义函数的并行处理

请帮帮我！！：（

它是执行parallelized（）函数时的输出。

MINC 201902020-11-03 08:15:23.469221

MINC 201901 2020-11-03 08:15:23.469228

MINC 201903 2020-11-03 08:15:23.469482

MINC 2019042002-11-03 08:15:23.469703

MINC 201905 2020-11-03 08:15:23.469915

MINC 201906 2020-11-03 08:15:23.470106

MINC 201907 2020-11-03 08:15:23.470283

值201902在PID中：1222

MINC 201908 2020-11-03 08:15:23.470459

值201901在PID中：1221

值201903在PID中：1223

值201904在PID中：1224

值201905在PID中：1225

值201906在PID中：1226

值201907在PID中：1227

值201908在PID中：1228

MINC 2019092020-11-03 08:15:23.489395

值201909在PID中：1222

Answer 1

这是代码的简化版本，您使用multiprocessing.Pool的方式似乎是正确的。如果子进程中的代码出现故障，程序可能挂起。您确定您的pd.read_sql代码以良好状态退出吗

import datetime
from multiprocessing.pool import Pool
from os import getpid

yms = range(1, 10000)


def minc(ym):
    print('MINC %s %s %s\n' % (ym, getpid(), str(datetime.datetime.now())))

    return ym


def parallelized():
    pool = Pool(processes = 8)
    df = pool.map(minc, yms)
    print("FINISHED")
    assert list(yms) == df

    # you don't really have to close/join but it is still a good idea
    # https://stackoverflow.com/a/38271957/598057
    pool.close()
    pool.join()

parallelized()