Skip to content

GitLab

  • Menu
Projects Groups Snippets
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • hyteg hyteg
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 36
    • Issues 36
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 3
    • Merge requests 3
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • hyteg
  • hyteghyteg
  • Issues
  • #141

Closed
Open
Created Nov 06, 2020 by Nils Kohl@kohl🌝Maintainer

PETSc KSPSolve() leaks memory when used with MPI

When used in parallel with MPI, PETSc's KSPSolve() apparently leaks memory.

While the amount might be small enough to ignore for stationary problems, it accumulates during time-dependent simulations with thousands of calls to KSPSolve().

Happens for both, preconditioned and 'vanilla' MINRES.

Memory allocation measured with rusage(), right before and after KSPSolve() gives some evidence:

[0][DEVEL   ]-----(48.045 sec) Before solve
[0][INFO    ]-----(48.045 sec) ========================= Memory Usage Info =========================
[0][INFO    ]-----(48.045 sec)  method: rusage
[0][INFO    ]-----(48.045 sec)                        +--------------+--------------+--------------+
[0][INFO    ]-----(48.045 sec)                        |          sum |          min |          max |
[0][INFO    ]-----(48.045 sec)  ----------------------+--------------+--------------+--------------+
[0][INFO    ]-----(48.045 sec)  allocated memory (GB) |        0.747 |        0.185 |        0.187 | 
[0][INFO    ]-----(48.045 sec)  ----------------------+--------------+--------------+--------------+
[0][INFO    ]-----(48.045 sec) =====================================================================
[0][INFO    ]-----(48.045 sec) 
[0][DEVEL   ]-----(48.460 sec) After solve
[0][INFO    ]-----(48.460 sec) ========================= Memory Usage Info =========================
[0][INFO    ]-----(48.460 sec)  method: rusage
[0][INFO    ]-----(48.460 sec)                        +--------------+--------------+--------------+
[0][INFO    ]-----(48.460 sec)                        |          sum |          min |          max |
[0][INFO    ]-----(48.460 sec)  ----------------------+--------------+--------------+--------------+
[0][INFO    ]-----(48.460 sec)  allocated memory (GB) |        0.752 |        0.187 |        0.189 | 
[0][INFO    ]-----(48.460 sec)  ----------------------+--------------+--------------+--------------+
[0][INFO    ]-----(48.460 sec) =====================================================================

I was, however, not yet able to detect this via Valgrind nor AddressSanitizer.

Notes:

  • it does not happen when run w/o MPI, i.e. in a serial run - so apparently some MPI resources are not handled properly inside PETSc
  • a similar issue is reported (but not solved) in the PETSc mailing list: https://lists.mcs.anl.gov/pipermail/petsc-users/2019-May/038776.html (reads like we are experiencing the same issue here)

Tested with:

  • HyTeG: 452ac8ca
  • PETSc version: 3.12.3
  • OpenMPI 3.1.5
Assignee
Assign to
Time tracking