Skip to content
GitLab

Streamed GPUPackInfo

Merged João Victor Tozatti Risso requested to merge (removed):gpupackinfo_stream into master

Refactored version of the StreamedGPUPackInfo which does not modify how the MPI buffers are allocated (i.e. does not uses allocators as in !46 (closed) . Using this approach maintains compatibility with MSVC compilers -- contrary to using allocators in the MPI buffers, and therefore is preferred to !46 (closed) .

Proposed changes:

  • CUDA streams: Add support for asynchronous (un)packing operations using CUDA streams in cuda::communication::GPUPackInfo. Through asynchronous operations it is possible to overlap GPU computation and MPI communication in simulations (e.g. LBM simulations). Asynchronous copies in CUDA require pinned memory on the host, and for that purpose a staging buffer is introduced (i.e. cuda::communication::PinnedMemoryBuffer) in the cuda module, which is used to stage data between the GPU and the MPI buffers.
  • zyxf layout: Add zyxf field layout support in GPUPackInfo through extensions of the functions in cuda::GPUCopy.
  • Extended GPUPackInfo test: Add stream and zyxf layout tests to the GPUPackInfoTest to test the proposed implementation.
  • Extended Kernel: add CUDA stream and shared memory configuration support in cuda::Kernel class

Closes #27 (closed), and also closes !46 (closed)

Edited by João Victor Tozatti Risso