Instable System

Setup, Configuration and Use Runtime
1

image
image1062×908 56.4 KB

Hi, I am running OH since 2020 and it was always working fine, but since the last few days (maybe a week) I am experiencing regular instabilities. It is not completely dead but seems like the rules stop working.
Yesterday I rebooted and upgraded the system but today it happened again.
I also note the CPU load is on high side.
Last change I did, is I added one device with about 100 Modbus registers (things).

Any ideas what to look for in the logs?

After restart I checked the running processes and the OH is taking the most resources 25-35%

image
image1184×788 49.6 KB

Thanks

2

How fast are you polling modbus and how often the items get values changed ? Maybe sd card cannot keep up

3

You might be right . I just checked the poll intrval and it is 600ms which is way too fast and unnecessary. I will change it to 60s.

It is a new device I started configuring last week but it is not 100% operational yet (free time issues).

FYI I have 128GB SSD connected via USB. I do not have SD card. Not sure if it is faster or slower than SD card

4

Well the only way to see if your SSD via usb can keep up is by installing iotop on your openhabian and look what happens when modbus is active. But normally if you don’t need the values so often to be changed then increase the time and also careful with persistence of those items can put a big strain.

5

Thanks for the heads up. Those 600ms were mistake. I increased the poll rate to 10s and I see the CPU load doesn’t go above 25%, so this might solve the problem.

6

It just happened again. I noted the log stopped logging item updates and no other error can be seen

2022-03-22 20:02:53.145 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'Memory_Used' changed from 1460 to 1461

2022-03-22 20:02:53.400 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'PANEL_MESSAGE' changed from 650: Partition can now be armed. to 609: General status of the zone - open.

2022-03-22 20:02:53.402 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'ZONE2_STATUS' changed from CLOSED to OPEN

2022-03-22 20:02:53.404 [INFO ] [hab.event.GroupItemStateChangedEvent] - Item 'grpDSCAlarmMotion' changed from CLOSED to OPEN through ZONE2_STATUS

2022-03-22 20:02:53.405 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'ZONE2_MESSAGE' changed from  to 609: General status of the zone - open.

==> /var/log/openhab/openhab.log <==

2022-03-22 20:03:09.506 [WARN ] [p.internal.http.HttpResponseListener] - Requesting 'http://192.168.4.221/real_time_data.xml' (method='GET', content='null') failed: Total timeout 3000 ms elapsed

2022-03-22 20:03:39.513 [WARN ] [p.internal.http.HttpResponseListener] - Requesting 'http://192.168.4.221/real_time_data.xml' (method='GET', content='null') failed: Total timeout 3000 ms elapsed

I can also see OpenHAB Java process is using almost 100% cpu

image
image1080×2400 342 KB

Not sure what to look for.
Any suggestions?

7

Once you get beyond non-trivial Modbus configurations, you should pay attention to consequences. It is quite possible to poll at 600mS periods, using methods to reduce knock-on workload.

Te usual effect of neglecting this is hammering the persistence service(s), which impacts system performance overall.

8

My Modbus polls are every 5 or 10 seconds now and I also deleted about 50% of the items that were in the original post.
Polling every 600ms would probably work since I have 4 slaves daisy chained.
But I do admit there is place to optimise the system so I will have a look in it.