使用Joblib进行多处理：对函数的一个参数进行并行化

Question

有没有办法让parallel函数接受多个参数，但只在其中一个参数上并行

假设我有一些代码：

def my_function(graph,graph_coefficients, thing_i_want_to_parallelise_over):

     <do_thing>

     return value 


results = Parallel(n_job2=2)(delayed(my_function(one_graph,graph_coefficients)(thing_i_want_to_parallelise_over) for thing_i_want_to_parallelise_over in range(1,3))

有办法做到这一点吗？有多个函数可以调用，因此执行一个简单的概括函数实际上不是一个选项

Answer 1

我不知道我是否理解您的问题，但您的格式不正确

您应该创建包含所有参数的元组

(one_graph, graph_coefficients, x) for x in range(1,3)   # args

然后你就应该把它和

delayed( my_function )

像

results = Parallel(n_jobs=2)( 
                delayed( my_function )
                (one_graph, graph_coefficients, x) for x in range(1,3)
          )

---

最终，您可以尝试使用lambda

lambda x: my_function(one_graph, graph_coefficients,x)

然后你可以使用

(x) for x in range(1,3)

像

results = Parallel(n_jobs=2)(
                delayed( lambda x: my_function(one_graph, graph_coefficients,x) ) 
                (x) for x in range(1,3) 
          )

---

或与functools.partial

partial(my_function, one_graph, graph_coefficients)

像

from functools import partial

results = Parallel(n_jobs=2)(
                delayed( partial(my_function, one_graph, graph_coefficients) ) 
                (x) for x in range(1,3) 
          )

---

最小工作代码

from joblib import Parallel, delayed

def my_function(graph, graph_coefficients, thing_i_want_to_parallelise_over):
    print('my_function:', graph, graph_coefficients, thing_i_want_to_parallelise_over)
    value = 2 * thing_i_want_to_parallelise_over
    return value 

one_graph = 'A'
graph_coefficients = 'B'

#   

results = Parallel(n_jobs=2)(
                delayed( my_function ) 
                (one_graph, graph_coefficients, x) for x in range(1,3) 
          )

print('results:', results)

#   

results = Parallel(n_jobs=2)(
                delayed( lambda x: my_function(one_graph, graph_coefficients,x) ) 
                (x) for x in range(1,3) 
          )

print('results:', results)

#   

from functools import partial

results = Parallel(n_jobs=2)(
                delayed( partial(my_function, one_graph, graph_coefficients) ) 
                (x) for x in range(1,3) 
          )

print('results:', results)