Is XML format suitable for big meshes in parallel ?

Question

Hi,

I am running computations in parallel on large meshes, that are read from XML format :

mesh = Mesh(meshname + ".xml")

and I get the error message :

*** Warning: XML file 'meshname.xml' is very large. XML files are parsed in serial, which is not scalable. Use XMDF/HDF5 for scalable IO in parallel

Is XML a proper format for parallel computation ? Is there a way within FEniCS to change mesh format into HDF5 ?

Many thanks in advance,

Claire

Comments

You can read in XML in serial, and write out HDF5 (as below)

Answer 1

HDF5 is the preferred format for large meshes (and other data).

Use it like this:

f = HDF5File(mesh.mpi_comm(), meshname+".hdf5", 'w')
f.write(mesh, meshname)

And re-read like this:

mesh = Mesh()
f = HDF5File(mpi_comm_world(), meshname+"hdf5", 'r')
f.read(mesh, meshname, False)

Comments

Thanks for your answer, does it also work for MeshFunction:

meshfunction = MeshFunction("size_t", mesh, meshname + "_meshfunction.xml")

with :

f = HDF5File(meshfunction.mpi_comm(), meshname+"_meshfunction.hdf5", 'w')
f.write(meshfunction, meshname_meshfunction)

and then :

meshfunction = MeshFunction()
f = HDF5File(mpi_comm_world(), meshname+ "_meshfunction.hdf5", 'r')
f.read(mesh, meshname, False)

?

---

Yes, it works with MeshFunction as well, but not exactly like that. You can store it to the same file as well if you want:

f = HDF5File(mesh.mpi_comm(), meshname+".hdf5", 'a') # Open in append mode
f.write(meshfunction, meshname+"/"+meshfunction_name) # Store it in a subgroup of meshname
f.close()

and read it back like:

mf = MeshFunction("size_t", mesh, dim)
f = HDF5File(mesh.mpi_comm(), meshname+".hdf5", 'r')
f.read(mf, meshname+"/"+meshfunction_name
f.close()

I advice you to have a look at the HDF5File documentation.

---

Thanks, I will ! Can I append 2 (or more) mesh functions as different subgroups of the same meshfile ?

---

Yes, you can store as much as you'd like to a single file. Functions, MeshFunctions, MeshValueCollections, Meshes etc.

A HDF5 file is structured like a file system, so just store the datasets (equivalent of files) in different groups (equivalent of folders). You can inspect a HDF5 file using the graphical tool hdfview, or the command line tool h5ls .