Openhab not responding, Java heap space error

Setup, Configuration and Use Runtime
1

Hi there,

after working for a long time, openhab stopped to work a few times last week.
The output of systemctl status openhab2.service is:

● openhab2.service - openHAB 2 - empowering the smart home
   Loaded: loaded (/usr/lib/systemd/system/openhab2.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2019-11-15 18:25:47 CET; 3 days ago
     Docs: https://www.openhab.org/docs/
           https://community.openhab.org
 Main PID: 487 (java)
    Tasks: 142 (limit: 4915)
   Memory: 839.8M
   CGroup: /system.slice/openhab2.service
           └─487 /usr/bin/java -Dopenhab.home=/usr/share/openhab2 -Dopenhab.conf=/etc/openhab2 -Dopenhab.runtime=/usr/share/openhab2/runtime -Dopenhab.userdata=/var/lib/openhab2 -Dopenhab.logdir=/var/log/

Nov 18 03:13:23 openhab karaf[487]: java.lang.OutOfMemoryError: Java heap space
Nov 18 03:13:31 openhab karaf[487]: Exception in thread "pool-3397-thread-1" java.lang.OutOfMemoryError: Java heap space
Nov 18 03:13:31 openhab karaf[487]: Exception in thread "pool-3399-thread-1" java.lang.OutOfMemoryError: Java heap space
Nov 18 03:16:03 openhab karaf[487]: Exception in thread "ESH-discovery-6685" java.lang.OutOfMemoryError: Java heap space
Nov 18 03:16:04 openhab karaf[487]: Exception in thread "pool-3402-thread-1" java.lang.OutOfMemoryError: Java heap space
Nov 18 03:23:06 openhab karaf[487]: Exception in thread "fileinstall-/var/lib/openhab2/etc" java.lang.OutOfMemoryError: Java heap space
Nov 18 03:23:34 openhab karaf[487]: Exception in thread "ESH-discovery-6696" java.lang.OutOfMemoryError: Java heap space
Nov 18 03:24:39 openhab karaf[487]: Exception in thread "Timer-1193" java.lang.OutOfMemoryError: Java heap space
Nov 18 03:26:54 openhab karaf[487]: Exception in thread "Timer-1194" java.lang.OutOfMemoryError: Java heap space
Nov 18 03:53:05 openhab karaf[487]: Exception in thread "ESH-thingHandler-138" java.lang.OutOfMemoryError: Java heap space
lines 1-21/21 (END)

I typed this and after a reboot the system is running so far.
export JVM_ARGS="-Xms1024m -Xmx1024m

The questions are:

  1. Will this solve this problem ?
  2. What values are recommended ?

System:
Raspberry Pi 4, 4GB
openHAB 2.4.0-1 (Release Build)
installed with openhabian 1.5

2

  1. Probably not, you most likely are running a binding that has a memory leak. It will likely just make it take longer before it runs out.

  2. The default settings are the recommended settings.

To solve the problem you need to find the binding that has the memory leak and file an issue on that binding. This will likely require trial and error.

However, it makes no sense to do the trial and error on a version of OH that is approaching a year old. You will want to upgrade to the latest milestone or snapshot version.

There is some evidence that the Nest binding may have a memory leak. I know of no other bindings that have been identified with this problem.

3

list your bindings please

4

Interesting and a pity at the same time. Here are my bindings:

46
Screenshot 2019-11-19 at 08.14.46.png1758×1612 242 KB

It can be said that there are the following Bindings with 1.x versions:

  • Exec Binding
  • Mosquitto Binding
  • System Info Binding

Obviously, they should be migrated to 2.x from me.

@rlkoshak,

  1. Can I find this binding with htop or something similar ?
  2. How to restore the default settings, exactly, which values do I have toenter in the “export” command above ?

So, you think it would be a good idea to upgrade from 2.4 stable to 2.5 milestone or snapshot ? From desire for a stable running system, I would prefer milestone.

5

  1. No. If you know how to use and interpret the results of a profiler, you might be able to pin point it but others have not been successful doing this. The only thing I’ve seen work is disabling bindings until the memory leak stops.

  2. You don’t really need to since your RPi 4 has plenty of RAM. openHABian configures it with -Xms250m -Xmx350m. I’m not sure what is there by default, maybe nothing.

If you want to seriously find the source of the problem, you need to be running on the latest version of the code so you are not diagnosing a problem that has already been fixed. Because 2.5 M5 was just released a few days ago, working with it would probably be fine.

6

Thank you for the answers. I just upgraded to 2.5 Milestone and uninstalled the first bindings. Let’s see what will happen.

7

When your system has been running for a while you could make a heap dump and analyze where all the memory is consumed with Eclipse Memory Analyzer Tool (MAT). If you configure the runtime such that it will always write a heap dump on OutOfMemoryErrors (OOM) you can always analyze them after they just occurred. I am very interested in the results.

  1. Create a heap dump from the console with the dev:dump-create command. This may take several minutes when you run it on a Raspberry Pi.

  2. Try adding EXTRA_JAVA_OPTS="-XX:+HeapDumpOnOutOfMemoryError" to your /etc/default/openhab2 file to get automatic heap dumps in case of an error.

See also: