from multiprocessing.pool import Pool
def my_fun(x):
return 2*x
input_list = [3,4,5]
num_of_processes = 4
Pool pool = Pool(num_of_processes)
result_list = pool.map(my_fun, input_list) # the second argument has to be a list
pool.close()
pool.join()
print result_list # should be [6,8,10]
It is simple and easy to understand: we set up how many processes we want, fit the function and input data to the processes.
What if I want to send one more argument into the function? Say, add a constant to the output (and the constant remain the same for different input x)
def my_fun2(x, general_const):
return 2*x + general_const
You can simply wrap the arguments if you wish. But I will introduce a fancier way to do this: use partial. See the code below
from multiprocessing.pool import Pool
from functools import partial
def my_fun2(x, general_const):
return 2*x + general_const
input_list = [3,4,5]
my_const = 1000
num_of_processes = 4
Pool pool = Pool(num_of_processes)
result_list = pool.map(partial(my_fun2, general_const=my_const), input_list)
## if you prefer, you can also separate them (just another layout, does not change anything)
# my_fun2_partial = partial(my_fun2, general_const=my_const)
# result_list = pool.map(my_func2_partial, input_list)
pool.close()
pool.join()
print result_list # should be [1006,1008,1010]
That's it!
