openHAB 3 runs out of memory / java heap space errors, CPU 100%+ after a few hours

Thanks for your advice @Sebi
I will try to disable LSP (but I use a lot VSC, i am annoyed).
You mean you stop the bundle or uninstall ? if you stop only it restarts automatically at next startup no ?
I spent so many hours to find this memory leak … I am now at a point where I commented out almost 100% of my rules and uninstall every bundle except one (zwave) … and still memory increase constantly.
Wether there is a memory leak in core (or my install) or I dont have the right approach to debug.

Just for your information, I am in the exact same situation…

Trying to find out what causes my OOMs by disabling bindings…

Above graph is from the last 7 days. The “spike” was a shutdown where things were totally out of control. The last four days seems quite fine though, after I disabled a specific binding.

I won’t draw any conclusions yet though.

I only disabled LSP in openHAB extension in VSCode, I didn’t disable or uninstall anything in openHAB. To disable LSP in VSCode go to Extensions, openHAB, Manage and do a search for Languageserver option. Uncheck the option Languageserver: Remote Enabled.
afbeelding
Check User and Workspace settings.

Has anyone opened a bug on github to track the new memory leak you are seeing in 3.1? I’ve been on the 3.1 snapshots for a long time and I haven’t seen a memory leak so I’m wondering what the difference between our setups is.

Repeatable “Out of memory” found regarding large image items. I’ll document it here for others who might find it helpful.

OpenHab 3.1.0 Running on Pi4 4gb.
Bindings:

  • Amazon Echo
  • Astro
  • Epson Projector
  • Exec (used extensivly)
  • Harmony Hub
  • Ical
  • iRobot
  • Network
  • Onkyo
  • OpenGarage
  • OpenHab Zigbee
  • Open Weather
  • Roku (enabled but not in use yet)
  • System Info
  • Z-Wave
  • ZoneMinder

Extensive Use of HabPanel

I have 9 Lenovo M8 and M10 Tablets running habpanel Viewer to display screens, controls, cameras, etc. I have lots of large images and video streams that run over my wifi network. Some of these image items get stored as OpenHab image items.

Repeatable trigger events: Updating image items too quickly WILL trigger Java Heap OOM events. Example 9 tablets and 12 Zoneminder cameras pushing high res image items to OpenHab every few seconds will normally function correctly. HOWEVER if my wifi is congested or performing poorly, The tablets and camera images will try to update more frequently than they can finish. This leads to a pileup of image items spawning and murdering the java heap.

A hacky work around I have is an exec item that grabs the current max java heap percentage, If it exceeds 50% the rule that updates the images is skipped for an iteration. If the heap exceeds 90% a restart/reload command is sent to my tablets. This clears out any image queues that have pilled up.

If anyone with this problem has lots of .js files, they might like to look at what I just posted on this thread, in case it’s relevant:

Hi all,
Just for information, I figured out the main problem for my memory leaks : ipcamera addon.
The plugin was the cause of tons of threads, file descriptors and memory leaks.
Interresting command also in console : shell:info
I completly removed the bundle, waiting for a serious upgrade. Until this upgrade, I re-code all the stuff of the plugin in the rules …
Perhaps it can help someone else.

4 Likes

maybe @matt1, who is afaik mainting the ipcamera binding, can work together with you tracking down the issues you have noticed in the binding?

A lot has changed in the binding recently, upgrade to the 3.2milestone3 or use the even newer version of the binding which can work on 3.1 Stable and can be downloaded here:

Index of /openhab/IpCameraBinding/ (pcmus.com)
If you get dependency issues, just install the tellstick binding.

There is a post on the forum about the breaking change that this version brings as the serving is now done on port 8080 for all cameras.

Thanks @matt1 !
I will give it a try next week.
I understand that your jar is 3.2-snapshot that can work over OH 3.1-stable.

Yes it will work on 3.1 Stable, just need to be aware of the breaking change of the new URLS. You can run with it and when you upgrade to milestone 4 or 3.2 Stable just go back to the built in binding and you have dealt with the breaking change already.

I would like to post my experience here because I had the “out of memory” problem with my openhab2 (2.5.12-1) installation.

Optimizing the JAVA_OPTS didn’t do anything for me. Instead, however, I found the dependency on VisualStudio code and the extension to be the cause of the problem.

As Andrew said, one should disable the language server in the settings of the openhab plugin for VScode.

Hope this post helps others who end up here via a Google search.

The funny thing was, that it worked without any problems running openhab on virtualbox. Moving VDI to RAW and importing to proxmox and running there brought the problem. Strange.

2 Likes

OH 3.2… I had this problem. Disabling vscode language server seemed to fix it. Also, I got frequent openHAB crashes when using ctrl c to copy code. Disabling the server seems to have fixed that too.

1 Like

LSP-issue still persists with OH 3.3.0.M5.
Is anybody aware of any activity in solving this issue?

1 Like

This post ( above ) shows the registered issue on github. According to the tracker it doesn’t look like there would be any current activity.

Hello everyone,

I’ve been fiddling with my new installation of oh 3.3 for a while now.
Every time the system crashes after a certain amount of time. I am now monitoring my heap.

I have seen that java uses 100% cpu as root. Is that normal?

I also still face this issue

Okay but this doesn’t cause crashes? How long have you been running your system?

@jelmer1 Can you check what process this is that is running java as root? In my install (openhabian), openhab java is running as the openhab user, not root.
Try ps -ef | grep java

1 Like

@Kai - Can you create an official stacked post with listed clear procedures (also for non linux experts) for these annoing issue that many many many users still to have randomly in their OH life?

  • How to give you right informations (like how to extract the JAVA thread dump)
  • Is there something special to look inside on it?
  • A list of the corrently known root causes that bring to cylcis crushes due to JAVA HEAP SPACE

I think at the moment this is the most annoying issue OH have…

Thank you in advance.
Nicola