schwimmbad | A common interface to processing pools | Architecture library

MPI and Python's multiprocessing are actually two quite different systems for running distributed processes.

• Multiprocessing is based on the principle that your main program starts new processes when needed. To reduce the overhead of starting a new process, you can create a pool of other processes at the very beginning and then use that pool. But even then, it is still your main program that spawns off all those other processes.
• MPI, on the other hand, is based on the idea that you run the same script in different processes and make these different copies talk to each other (MPI is a messaging interface). The different processes are neither started by, nor under direct control of any individual copy of your program. Rather, you have an external manager mpirun for that.

The graphic below shows the difference: MPI on the left has a manager that starts four equivalent copies of the same script—each with a different rank to distinguish them by. Multiprocessing on the right has a main program that then spawns off other instances. Accordingly, multiprocessing can be run just like a normal Python script and does not require something like mpirun.

To come back to your question: when using multiprocessing, the main script can send off tasks to other processes. Even in the case of a pool: unless one of those secondary processes receives something to do, they usually just wait. With MPI, each process just works through the same program in parallel and each process acts as if it is the only process running the script. Here, you use the process's rank to distinguish what they are doing, and you use communication to synchronise their action and processing.