PETSc KSPSolve() leaks memory when used with MPI
When used in parallel with MPI, PETSc's KSPSolve() apparently leaks memory.
While the amount might be small enough to ignore for stationary problems,
it accumulates during time-dependent simulations with
thousands of calls to KSPSolve().
Happens for both, preconditioned and 'vanilla' MINRES.
Memory allocation measured with rusage(), right before and after KSPSolve() gives some evidence:
[0][DEVEL ]-----(48.045 sec) Before solve
[0][INFO ]-----(48.045 sec) ========================= Memory Usage Info =========================
[0][INFO ]-----(48.045 sec) method: rusage
[0][INFO ]-----(48.045 sec) +--------------+--------------+--------------+
[0][INFO ]-----(48.045 sec) | sum | min | max |
[0][INFO ]-----(48.045 sec) ----------------------+--------------+--------------+--------------+
[0][INFO ]-----(48.045 sec) allocated memory (GB) | 0.747 | 0.185 | 0.187 |
[0][INFO ]-----(48.045 sec) ----------------------+--------------+--------------+--------------+
[0][INFO ]-----(48.045 sec) =====================================================================
[0][INFO ]-----(48.045 sec)
[0][DEVEL ]-----(48.460 sec) After solve
[0][INFO ]-----(48.460 sec) ========================= Memory Usage Info =========================
[0][INFO ]-----(48.460 sec) method: rusage
[0][INFO ]-----(48.460 sec) +--------------+--------------+--------------+
[0][INFO ]-----(48.460 sec) | sum | min | max |
[0][INFO ]-----(48.460 sec) ----------------------+--------------+--------------+--------------+
[0][INFO ]-----(48.460 sec) allocated memory (GB) | 0.752 | 0.187 | 0.189 |
[0][INFO ]-----(48.460 sec) ----------------------+--------------+--------------+--------------+
[0][INFO ]-----(48.460 sec) =====================================================================I was, however, not yet able to detect this via Valgrind nor AddressSanitizer.
Notes:
- it does not happen when run w/o MPI, i.e. in a serial run - so apparently some MPI resources are not handled properly inside PETSc
- a similar issue is reported (but not solved) in the PETSc mailing list: https://lists.mcs.anl.gov/pipermail/petsc-users/2019-May/038776.html (reads like we are experiencing the same issue here)
Tested with:
- HyTeG: 452ac8ca
- PETSc version: 3.12.3
- OpenMPI 3.1.5