The idea of the this draft PR is to add an exporting functionality for OpenVDB (https://www.openvdb.org/). OpenVDB has several advantages over Paraview files.
- sparse octree data structure for files and in-memory representation
- data compression
- easy inclusion of meta-data
Blender 2.83 will be able to import OpenVDB (for now its in the experimental branch new-object-types https://builder.blender.org/download/branches/). So you can generate fancy cinematic animations from your simulation, visualizing different quantities with different shaders (https://www.youtube.com/watch?v=N4YMYGDEcPM).
Viewing in Paraview is planned for next release and can be done currently by using a simple Python conversion wrapper (so you can still double-click on a file an view it in Paraview).
I chose an naive approach for now. Every node just outputs its blocks in a separate file and copies its data first in OpenVDB s data structure. This has the advantage that we can use file compression. Fusing of the files can be done with a simple script in post-processing. Volumes will be non-overlapping and boolean operations to fuse different files with OpenVDB is a fast operation. OpenVDB has also a very low level IO system, it would also be possible to output the data directly in one uncompressed MPI file.
I'm using these files since over 1.5 years because I wanted to save disk space and have faster IO on my local machine. But I will be on vacation for the next 6 weeks. So just putting it here for information. Everything is still very WIP but should be working.