I would like to post my experience here because I had the “out of memory” problem with my openhab2 (2.5.12-1) installation.
Optimizing the JAVA_OPTS didn’t do anything for me. Instead, however, I found the dependency on VisualStudio code and the extension to be the cause of the problem.
As Andrew said, one should disable the language server in the settings of the openhab plugin for VScode.
Hope this post helps others who end up here via a Google search.
The funny thing was, that it worked without any problems running openhab on virtualbox. Moving VDI to RAW and importing to proxmox and running there brought the problem. Strange.