how to go parallel?

Question

With FEniCS, I have achieved satisfactory results on 2D Navier-Stokes equations based optimization. Now I want to go parallel with 3D problems.

I have access to two types of high performance computing resources: high-throughput computing and high performance computing clusters. But I don't know where to start.

1. Should I use high throughput computing or high performance computing?

2. Do I have to install from source or can I use binary installation and still run it on the clusters? I tried "fenics-install.sh". It didn't go very far before it gives me all kinds of errors.

Any tutorial on Parallel Computing with FEniCS is greatly appreciated. Section 10.4.1 in the book is too thin. We need an equivalent of The FEniCS Tutorial for Parallel FEniCS.

Answer 1

1. You probably want to go with HPC, but that depends on your problem. If you have one small problem that you want to solve sequentially for a wide range of parameters, then I guess HTC is the way to go. If you want to solve large problems by creating one large mesh that is subsequently decomposed between a large number of processors, then you need HPC.

2. You can generally use binary installations, but on clusters I think source installation is the way to go.

Almost all Fenics demos run in parallel without modification using mpirun. For example

mpirun -np 2 python demo_poisson.py

There is not much more to it, unless you want to dig deep and mess with dofs directly. assemble, solve, and most other functions work collectively with the same call as for the single-processor case.

Comments

Thanks for your answer.

1. Can you elaborate on why source installation is recommended on clusters?

2. How many times of speed-up can one expect to gain over serial processing in solving 3D Navier-Stokes equations at Re=50 ~ 1500? Assuming we have up to 16 nodes and 20 cores per node and network speed of 56 Gbit/s Infiniband. Assuming we are using 1 million Taylod-Hood elements.