Apply_async

In situations where we need parallel execution of our tasks, this is the function we'll need to use in order to submit tasks to our pool.

In this example, we are going to use a for loop to submit four tasks to our processing pool as with our newly discovered apply_async. We keep track of these tasks by appending them to a tasks array, and then iterate over this task array to wait for the results:

from multiprocessing import Pool
import time
import os

def myTask(n):
    print("Task processed by Process {}".format(os.getpid()))
    return n*2

def main():
    print("apply_async")
    with Pool(4) as p:
        tasks = []

        for i in range(4):
            task = p.apply_async(func=myTask, args=(i,))
            tasks.append(task)

        for task in tasks:
            task.wait()
            print("Result: {}".format(task.get()))

if __name__ == '__main__':
    main()

Upon execution of this, you should see that our four tasks are picked up by our process pool, and executed by four distinct processes. Because we call task.wait() in the same order that they are submitted to the tasks array, we then get an ordered output of the results printing out to our console:

$ python3.6 21_applyAsync.py
apply_async
Task processed by Process 99323
Task processed by Process 99324
Task processed by Process 99322
Task processed by Process 99325
Result: 0
Result: 2
Result: 4
Result: 6