Openhab 4.0.1 System unstable from "class org.openhab.core.internal.items.ItemUpdater"

I don’t think zram should affect load or, if it does affect load it’s going to make the load less rather than more because RAM is way faster than disk IO. But load isn’t specific to file systems. Load can be raised based on other things like network, serial devices, or really any resource that only one process/thread is allowed to access at a time.

I don’t have an RPi running OH right now but even a load of 1.8, particularly when not running OH, seems pretty high. What else do you have running on this machine? If it’s pure openHABian the load should be well under 1 with OH not running.

its debian on rpi4/8GB and docker install. No Openhabian. On docker running openhab, influxdb, Ngxinx-Reverse-Proxy, Pihole, Unifi, Grafana with mimir and so on, freepbx, Gitea are the most used Containers.

Openhab reads from serial port the smartmeter which sends i guess about 30 seconds some metercodes to openhab. and knx is via network router connected and the most senders on knx are changes of currents (about 15 measurements, which sends on 5% change to bus) Humidities, temperatures too (about 10). The rest of the knxstates are not changing much often.

Just got out a dns-problem from docker-daemon and fixed it, but seemed no effect to have on load

There’s more! :scream:

Even on an RPi 4 that’s probably too much for one machine. Unless and until you can get that load under control I would not expect this problem to be fixed. There is nothing that can be done from the OH side of things.

My first suspect as a problem causer is freepbx which, if it’s anything like Asterisk could consume a number of resources at a high priority.

If you want OH to run properly, you need to move stuff off this machine or move OH to another machine.

ok, thanks a lot for your help. i will try to get the load down or bye another rpi4/8GB (hopefully that its again available in the world)

Hi Andreas, did you manage to figure it out? similar problem. Clean install of Openahab 4.0 from SD card image provided by Openhab
updated to 4.0.1 release and have restored my previous install.

Release = Raspbian GNU/Linux 11 (bullseye)
Kernel = Linux 6.1.21-v8+
Platform = Raspberry Pi 4 Model B Rev 1.4/8gb

after about a day it freezes (will check the memory question when it crashes next)

Hi @alexkarageorgis ,
we have exactly the same Kernel and release running. So im not alone with my problems.
We figured out that it seems that there i a lot of load which could make the Problems.
So what i figured out in the meantime is that : https://community.openhab.org/t/openhab-4-0-1-and-network-pingdevice-on-docker-install-results-in-massive-threads/148658
So im pinging my network-devices to get the info if they are online or not. Openhab is making each ping-profile in a result of about 20 threads of arping per profile because it wants to ping the device from each existing bridge network which exists on the host. But i have this time no solution to only ping it from main network. You can see that if you open an interactive shell into container and make an “ps aux” and repeat this sometimes.

Maybe other impacts existing too but i did not figured out more.

So please let me know what you get out.

while I get the general idea of what you are saying I am not using a docker image. the only thing i can think of causing network traffic is the ip camera binding. - i will let it crash one more time (i am loosing about 1% of memory every hour or so.) and then disable it and see if that fixes things

Thanks Andrea, will keep a look on your post.

A

im using the ip camera binding too (with 2 cams)

on my system load seems more problem than memory. So possibly we have different problems but maybe some the same. future will show what we learn…

IP camera always increased my system load a fair bit. in any case i disabled it and will wait and see if the issue comes back.

Alex

today i got the load down:


The only thing i did is disabling the about 10 ping-checks profiles from network binding. Now it is the same load as if openhab is not running. But its still running.
I have opened another thread for the network binding: Openhab 4.0.1 and Network Pingdevice on Docker-Install results in massive threads

Now that the load is down, have the EventHandler warnings gone away?

Hi Rich,
the last days before my change today with disabling network-binding i also did not had such en “explosion” with thousands of there messages.
It could also be that some conversions from older ecma-scripts to actual javascript had some errors which caused them. Possibly. I guess i have now all rules actualized that they dont raise errors and i guess it should not run rules any more on the nashorn engine which i installed directly after upgrade to run the most rules.

Another person told me about his impacts of ip camera binding which im also using. But disabling did not gave me a hint of impacts of this.

Thats why i logged interactively to container and had a look at the threads which are running and thnking about them.

These very much threads of arping gaves me a hint to disabling the network binding. Then it lowers the load completely as i have shown in graph. So what we have seen and seems also some years before someone got out (linked the other cases) is that pinging via network-binding is not useful if you are using it in docker container. It makes absolutely no sence that it starts pingthreads from each network which is existing in docker when in openhab is defined take only the network XXX.
I guess there would be a little change nessecary to avoid the unnessecary pings.
For me, if i would need it in future this means to make a little shellscript and start that instead of network-binding.

So fazit: load can be impacted from openhab itself also if someone tells it isnt :wink:

I believe you already filed an issue but if not, make sure you do so. The network binding should not do that.

But I also recommend using the best tool for the job. OH is not a very good IT system monitoring system. If you need to monitor a bunch of devices and services via ping, you might be better off deploying a system like Zabbix, ELK stack, Prometheus, etc.

…i did a question in community about the network-binding and its behavior(no answers) ,
and also filed an issue now for the massive threads.
“best tool for the job”: i did use it because it was an easy way for showing which device running and which is not so other people in family can easily know which device to reboot :wink:
im also/already using prometheus/ LGTM and will use more this tool for that.

Today in morning it crashed again with WARN ] [ab.core.internal.events.EventHandler] - The queue for a subscriber of type 'class org.openhab.core.internal.items.ItemUpdater' exceeds 5000 elements. System may be unstable.
Im looking further if i can get out something. Maybe a systemlimit like open files or others…

@Scriptwriter just updating that since i unistalled the ip camera binidng i havent (yet) crashed - might be connected, maybe i will just get it later on.

Alex

just to put this down also. Zwave module / seems to be acting up also. i receive multiple same data from thermostats and energy meters. i.e. microsecond apart the same data 20 times. could be causing load issues. also send commands dont seem to work. another strange behaviour on my setup.

Hi @alexkarageorgis ,
did you realized that there is a problem of excessive update-logging in 4.0.0 and 4.0.1-versions? See https://community.openhab.org/t/successful-openhab-upgrade-v3-4-3-to-v4-0-1-potential-stumbling-blocks/148174 if the nessecary change in log4j2.xml is applied on your site. You can also do update to 4.0.2 which automaticly applies this change.

Thanks @Scriptwriter for pointing in the right direction. did the upgrade to 4.0.2 but not change to the logging. will try your two lines of code then

Strange.

** update. the file 4j2.xml contains one of the two lines of code -
not this one

   <Logger level="ERROR" name="openhab.event.GroupStateUpdatedEvent"/>

but i dont think this is the issue (see image below)

trying to add with Winscp as i am remote to the device currently but permission to edit/delete or overwrite is denied. is there a smart way to do this?

Alex

copy the file e.g. to /tmp folder.
then login via ssh and change to user root ( sudo bash )
from that shell you can copy the file from /tmp folder to the target folder
make sure that ownership and permissions are the same after this

1 Like

thanks @Wolfgang_S appreciate the sharing of cleverness - do you think given the missing code it will correct the issue?
this code exists already

 <Logger level="ERROR" name="openhab.event.ItemStateUpdatedEvent"/>

this part is the only missing part

<Logger level="ERROR" name="openhab.event.GroupStateUpdatedEvent"/>