What is the simplest way to use MPI?

Question

I have a growing piece of (Python) software which at some points solves a PDE using FEniCS. I now would like to solve these using a parallel solver (PETSc is the current backend). The documentation on MPI + FEniCS seems very scattered, but I read everywhere that it is very well supported. Hence the question from a MPI-novice: what is the recommended/simplest way (i.e. least modifications to the code) to solve everything in parallel? (version 1.4)

Thanks a lot.

Comments

(Unlike the examples that consist of one python script, I suppose I cannot just run mpirun -np 2 python whole_program.py)

Answer 1

did you try running the code using (here np is the number of procceses)

mpirun -n np python demo.py

?

Comments

I guess I just added a comment on my own question while you were answering :-)
As I understand, this command will run demo.py np times. In my case, demo.py would be the initial pythonscript that start the whole program execution, including the generation of meshfiles and other inputfiles etc. I would like to avoid doing this multiple times, i.e. only do the PDE-solving in parallel. I am not sure if what I am saying is clear, since I am new to MPI, but I was hoping for something as easy as a solver parameter option I could change.

(btw is it possible version 1.6 uses MPI by default if possible?)

---

The command above should run demo.py in parallel, but perhaps you should take a look at this for some background:

http://fenicsproject.org/qa/3025/solving-a-problem-in-parallel-with-mpi-%26-petsc

Basically, you tell python to run with MPI and FEniCS steps in to handle the rest of the things FEniCS does. If you use normal python commands, such as 'print', you see something printed np times since that command is not handled by FEniCS.

The best way to explore this in my experience is to play with it a little. Running in parallel is a great way to handle larger programs, although you will have to consider solver choices, etc.

---

I somehow feel that is should be possible to run Python as usual, and only use MPI for the parts related to FEniCS, rather than having to rewrite all the rest of my code so it is only executed once?

---

I don't know if is possible to do what you're saying, but if your whole_program is divided in multiple scripts maybe a solution for you is to create a bash code (e.g. run.sh) with the lines (in the linux way):

#!/bin/bash
python script1.py
python script2.py
mpirun -n 8 python solve_pde1.py
python script3.py
... etc

in this manner, if you run the bash code, the python scripts will be executed with the quantity of processes that you prefer (i don't have tested if this works).

---

That is a solution for sure, but all my PDE's are classes in Python, and I don't plan to rewrite everything to use MPI.

What I find strange is that on my computer at home, MPI seems to be used by default. That is to say, it works a lot faster than at work, using the same processor. When I interrupt the Python process, I get an error message saying:

An MPI process is aborting at a time when it cannot guarantee that all
of its peer processes in the job will be killed properly.  You should
double check that everything has shut down cleanly.

So is this because I use version 1.6 instead of 1.4, or were some compilation options different (FEniCS PPA for 1.6 and Anaconda for 1.4)? Would be interesting to know.