Get access to entire vector in parallel?

Question

Hi!

I need to access an entire array on each process when running my program in parallel, not only the local part of the array. What would be the easiest way to do this? I assume that some broadcasting between processes may be required...?

Joakim

Answer 1

Hi! The class Vector implements the method gather, maybe it can help.

Answer 2

You can use dolfin's MPI class just to do simple MPI stuff in Python. If you need more there is, e.g., the mpi4py module that I have used occasionally to do more advanced send, broadcast, gather, allgather (that appears to be what you want), etc. It works well with numpy arrays. Remember that communication is expensive and you should try to keep it at a minimum.

The Vector.gather can also be used to distribute the vector across processes. The following code mpitest.py

from dolfin import *
from numpy import array

mesh = UnitSquareMesh(2, 2)
V = FunctionSpace(mesh, "CG", 1)
u = interpolate(Expression("x[0]"), V)

x = Vector()
u.vector().gather(x, array(range(V.dim()), "intc"))

print MPI.rank(mpi_comm_world()), x.array() 

executed with two cpus leads to
[mikael@ubuntu ~]$ mpirun -np 2 python mpitest.py
Number of global vertices: 9
Number of global cells: 8
1 [ 1. 0.5 0. 0. 0. 0.5 0.5 1. 1. ]
0 [ 1. 0.5 0. 0. 0. 0.5 0.5 1. 1. ]

So the entire vector is in x on both cpus, which I think is what you requested.