I have a problem with my openHAB system. I switched from 2.5 to 4.1. The openHAB crashes regularly. There is no direct connection to rule or item update activities. As recommended, I switched from an RPi3 to an RPi4. But that didn’t improve the behavior. The only anomaly I see in the log files are warning just before the crash:
[WARN ] [ab.core.internal.events.EventHandler] - Dispatching event to subscriber ‘org.openhab.core.thing.internal.CommunicationManager@144e03fd’ takes more than 5000ms.
Overall, I see difficulties when connecting to the MainUI, BasicUI, etc. When attempting to access, the above error message occurs frequently. After several unsuccessful connection attempts (e.g. via the MainUI), the following warnings appear:
[WARN ] [ab.core.internal.events.EventHandler] - Dispatching event to subscriber ‘org.openhab.automation.jrule.internal.events.JRuleEventSubscriber@3787a9cb’ takes more than 5000ms.
Shortly afterwards, openhab restarts…
The RPi itself doesn’t seem to crash, just the OpenHAB system. According to the resource information, there is plenty of RAM and CPU capacity available.
Here is some information about the system:
RPi:
RPi4 - Model B
4GB RAM
4GB swap memory
32GB SD card
Additional Applications:
MQTT Broker
MariaDB
OpenHAB:
Version 4.1.1
file-based configuration
AddOns:
Basic UI
openHAB Cloud Connector
Exec Binding / EXEC Tranformation
System Info Binding
Hue Binding
Telegram Binding / Telegram Actions
Max! Binding
MQTT Binding / MQTT Actions
JavaScript Scripting
Regex Tools / Regex Transformation
JSON Tools / JSONPath Transformation / Binary To JSON transformation
Collection Of common blocks
JRule - openHAB Rules using Java
Iconify icon provider
JDBC Persistence MariaDB
Does anyone have an idea how I can isolate the problem?
This is (as you can see from your warnings) a very general issue and hard to pin down. However, in the past, one very common culprit for RPi users has been a SD card that is nearing the end of its life and having read/write issues.
I would first make sure you have a reliable back up and, second get a new card to see if that fixes the issue before you try anything else more drastic.
Do you use Zram? Is that full just before getting errors (try df in RPi-terminal)? What is the processor load, is that suddenly constant high, due to swapping (try htop in RPi-terminal)?
I just upgraded from RPi3 wirh 1GB to a RPi4 with 8GB for same reason.
On what interval do you experience these crashes. Did you already try to run a while without persistence and/or some bindings and if not successfully swap the disabled bindings?
=> SDCard:
I used a new 32GB SDCard from SanDisk (SanDisk Ultra). I would say it can’t be that…
=> RPi3 / RPi4
With Openhab 2.5 I used an RPi3 Model B+ with 1GB RAM. I then noticed (via System Info Binding) that the 1GB RAM is always used at 100% and the swap memory at 75%. OpenHAB is crashed once per day. Then I switched to RPi4 Model B 4GB RAM. Here the RAM utilization fluctuates between 25…35%. Swap is 100% free. Despite this improvement, openHAB crashes once per day at different times of the day
=> Crash Behavior
When openHAB is about to crash, I usually notice it because the BasicUI or MainUI is sluggish or no longer available. However, the LogViewer is still available without any problems. So I can watch the crash live via the LogViewer. I don’t see the CPU/RAM load increasing much either. You can see the warnings in the LogViewer
=> RPi - OS & ARCH
I used the official openHAB image as the operating system as version 4.0.1 and upgraded it to 4.1.1 2weeks ago: Raspberry Pi - 4.0.1
As written above: when I read the values via System Info, the CPU, memory, swap, storage are unremarkable. I added a new heap, but the heap also seems ok, it usually seems to be >70% free. What is noticeable is LOAD of the OpenHAB process. The LOAD fluctuates greatly between 5% and 90% without me doing anything. When I call MainUI and BasicUI (especially BasicUI with charts), LOAD sometimes increases to >220% (peak). Used size and threads are almost stable. I can’t see the peaks of “Current Process Load” in the “CPU Load”…
Hello again,
I did some further investigations. I changed the memory size for java as Kevin recommend. It helped a lot but the system is still running out of memory. Before this change there was a reboot once a day, now every fifth day. Memory (RAM, Heap) is increasing constantly. Swap Memory is not used at all…
As you have only one important unofficial package, JRule, check the version you installed was built with OH core version 4.1. That would be the first thing I would try to remove to see if it solves the issue.
Also the iconify icon provider.
Can you try using the binding from the marketplace linked below and tell us if it logs any WARN messages?
What makes you think you are running out of memory? Used memory does not mean what you probably think it does, use the binding below and it will log into DEBUG level the amount of ram that you actually have left in plain English.
