Efficient way to compute the pointwise data of a Matrix times a difference of a vector (with MPI)?

Question

Given construction of mass matrix $M_{c}$ (with entries $m_{ij}$) and some diffusion operator $D$ (with entries $d_{ij}$) using assemble() and some data from functions called $b$ and $c$ on a 2D structured mesh with CG1 elements, I would like to compute the following pointwise:

$$ m_{ij} ( b_{i} - b_{j} ) + d_{ij} ( c_{i} - c_{j} ) $$

I know in parallel with a PETSc LA backend, the matrix is split in a row-oriented way and the tricky part will be accessing the values of $b$ and $c$ that are not locally owned. I'm also aware that looping over these entires is probably not the most efficient either.

Any suggestions would be helpful!

Thanks!

Comments

Maybe more details could help to have a clearer picture of your problem. Mean while, this reference seems helpful: https://fenicsproject.org/qa/10386/impossible-operations-on-matrices-objects-a-b-a-t

Best!

---

Comment edited as an answer below.

Answer 1

I have determined a solution for this using a combination of the GenericDofMap's Local to Global Map and petsc4py's setValuesLocal() to set the sparse matrix entries using local indices and getValues with the local-to-global indexing to grab the values from mij and dij.

I also needed to gather ghost values using PETSc.Vec's updateGhosts() to compute the differences $c_i - c_j$ and $b_i - b_j$ since the column index may not be owned.

If anyone would like a code snippet example later, I'd be happy to provide it.