Question

• Documentation
• Tutorial - kerndel density esitmation with multiprocessing

import multiprocessing

num_procs = multiprocessing.cpu_count()
num_procs

4

def pi_multiprocessing(n):
    """Split a job of length n into num_procs pieces."""
    import multiprocessing
    m = multiprocessing.cpu_count()
    pool = multiprocessing.Pool(m)
    results = pool.map(pi_cython, [n/m]*m)
    pool.close()
    return np.mean(results)

For small jobs, the cost of spawning processes dominates

n = int(1e5)
%timeit pi_cython(n)
%timeit pi_multiprocessing(n)

100 loops, best of 3: 1.95 ms per loop
10 loops, best of 3: 32.6 ms per loop

For larger jobs, we see the expected linear speedup

n = int(1e7)
%timeit pi_numpy(n)
%timeit pi_multiprocessing(n)

1 loops, best of 3: 718 ms per loop
10 loops, best of 3: 148 ms per loop

Not all tasks are embarassingly parallel. In these problems, we need to communicate across parallel workers. There are two ways to do this - via shared memory (exemplar is OpenMP) and by explicit communication mechanisms (exemplar is MPI). Multiprocessing (and GPU computing) can use both mechanisms.

See MOTW for examples of communicating across processes with multiprocessing.

Using shared memory can lead to race conditions

from multiprocessing import Pool, Value, Array, Lock, current_process

n = 4
val = Value('i')
arr = Array('i', n)

val.value = 0
for i in range(n):
    arr[i] = 0

def count1(i):
    "Everyone competes to write to val."""
    val.value += 1

def count2(i):
    """Each process has its own slot in arr to write to."""
    ix = current_process().pid % n
    arr[ix] += 1

pool = Pool(n)
pool.map(count1, range(1000))
pool.map(count2, range(1000))

pool.close()
print val.value
print sum(arr)

500
1000