如何提高Python中调用400次函数的速度

2024-06-28 15:16:15 发布

您现在位置:Python中文网/ 问答频道 /正文

我有一个名为dfs的列表。它包含大小为700 rows x 400 columns的400Pandas dataframes

我有这样一个函数:

def updateDataframe(i):
    global dfs
    df = dfs[i]

    df["abc"].iloc[-1] = "xyz"

    df["abc2"] = df["abc"].rolling(10).mean()

    ........ #More pandas operations like this

    dfs[i] = df



for i in range(len(dfs)):
    updateDataframe(i)

现在,这个循环需要10秒来执行。我尝试过python多处理,但它需要同样的时间,甚至更多的时间

我尝试过的事情:

import multiprocessing.dummy as mp #Multi process Library, used for speeding up download
p=mp.Pool(8) #Define Number of Process to Use
p.map(updateDataframe,range(len(dfs))) # Call the Download Image funciton
p.close() #Close the multi threads
p.join()

我也试过:

from multiprocessing import Process

if __name__ == "__main__":  # confirms that the code is under main function
    processes = []
    for i in range(len(dfs)):
        process = Process(target=updateDataframe, args=(i,))
        processes.append(process)
        processes.start()

    # complete the processes
    for i in range(len(processes)):
        processes[i].join()

Tags: theinimportdfforlen时间range