使用多处理的Python池.map但这并不能加快计算速度

2024-09-27 23:20:48 发布

您现在位置:Python中文网/ 问答频道 /正文

我正在尝试使用多处理来加速我的计算池.map在HPC上。 但是当代码运行时,它并没有大大加快速度。 我已经隔离了程序的主要功能,并做了一些案例研究。你知道吗

def foo1(density):
    start = time.time()
    gs = GeneralSolver(angle, angle, 1, plies_orientation, ply_height, stiffness_matrix, density, num_nods,  f_min, f_max, nb_f)
    k_mat_SAFE, a0_t_mat, s0, s0_t_mat, sh0, sh0_t_mat = gs.get_dispersion_curves(f_min, f_max, nb_f)
    return (time.time() - start)

if __name__ == '__main__':   


    p=[2.7, 2.4, 2.5, 2.5]

    _start = time.time()
    pool = mp.Pool(processes=2)
    var_returned1=pool.map(foo1, p) #sw.evaluation1_parallel
    pool.terminate()
    print( "RESULT parallel", var_returned1)
    print( "TOTAL EXECUTION TIME parallel", (time.time() - _start))

    _start = time.time()
    var_returned2=[]
    for ii in range(len(p)):
        var_returned2.append(foo1(p[ii]))
    print("RESULT serie", var_returned2)

    print( "TOTAL EXECUTION TIME serie", (time.time() - _start))

当我在笔记本电脑上尝试此代码(4核8 Gb RAM,但我只使用2核进行多处理)时,得到以下结果:

RESULT parallel [8.770999908447266, 5.209000110626221, 4.703000068664551, 5.190000057220459]
TOTAL EXECUTION TIME parallel 14.720999717712402
RESULT serie [4.5350000858306885, 4.534999847412109, 4.503999948501587, 4.526000261306763]
TOTAL EXECUTION TIME serie 18.1010000705719

当我在HPC上试用时(使用了2个内核,分配了160 Gb的RAM),我得到:

RESULT parallel [11.814664125442505, 11.850022315979004, 10.135426998138428, 10.130557537078857]
TOTAL EXECUTION TIME parallel 22.07818293571472
RESULT serie [4.681218147277832, 4.565039157867432, 4.637110471725464, 4.539891958236694]
TOTAL EXECUTION TIME serie 18.423360109329224

当我在输入列表(109个元素)和HPC上的更多内核(10个内核和160 Gb RAM)中尝试时,我得到:

RESULT parallel [39.19079065322876, ..., 31.59877610206604]
TOTAL EXECUTION TIME parallel 375.15932393074036
RESULT serie [4.305459260940552, ..., 4.620036363601685]
TOTAL EXECUTION TIME serie 501.7964723110199

为什么要在HPC上运行这么长时间? 我能做些什么来加速我的计算?你知道吗

谢谢


Tags: timeparallelvarresulthpcstarttotalprint

热门问题