Mem issues with OH4

Since the update to OH4, I have been experiencing problems that lead to full memory utilisation. As a result, I often see Java heap space out of memory error messages and the entire system seems to hang (GUI unresponsive, no reaction on updated MQTT items).
Even -Xmx2048 in EXTRA_JAVA_OPTS is not enough.
The problem is probably related to the fact that I now very often see duplicates in the EventUpdates in the log:

2023-07-21 21:01:49.615 [INFO ] [openhab.event.ItemStateUpdatedEvent ] - Item 'OG_Bad_Temperatursensor_Batterieladung' updated to 68
2023-07-21 21:01:49.616 [INFO ] [openhab.event.ItemStateUpdatedEvent ] - Item 'OG_Bad_Temperatursensor_Batterieladung' updated to 68
2023-07-21 21:01:49.616 [INFO ] [openhab.event.ItemStateUpdatedEvent ] - Item 'OG_Bad_Temperatursensor_Batterieladung' updated to 68
2023-07-21 21:01:49.616 [INFO ] [openhab.event.ItemStateUpdatedEvent ] - Item 'OG_Bad_Temperatursensor_Batterieladung' updated to 68
2023-07-21 21:01:49.616 [INFO ] [openhab.event.ItemStateUpdatedEvent ] - Item 'OG_Bad_Temperatursensor_Batterieladung' updated to 68
2023-07-21 21:01:49.616 [INFO ] [openhab.event.ItemStateUpdatedEvent ] - Item 'OG_Bad_Temperatursensor_Batterieladung' updated to 68
2023-07-21 21:01:49.616 [INFO ] [openhab.event.ItemStateUpdatedEvent ] - Item 'OG_Bad_Temperatursensor_Batterieladung' updated to 68
2023-07-21 21:01:49.616 [INFO ] [openhab.event.ItemStateUpdatedEvent ] - Item 'OG_Bad_Temperatursensor_Batterieladung' updated to 68
2023-07-21 21:01:49.616 [INFO ] [openhab.event.ItemStateUpdatedEvent ] - Item 'OG_Bad_Temperatursensor_Batterieladung' updated to 68
2023-07-21 21:01:49.616 [INFO ] [openhab.event.ItemStateUpdatedEvent ] - Item 'OG_Bad_Temperatursensor_Batterieladung' updated to 68
2023-07-21 21:01:49.617 [INFO ] [openhab.event.ItemStateUpdatedEvent ] - Item 'OG_Bad_Temperatursensor_Batterieladung' updated to 68
2023-07-21 21:01:49.617 [INFO ] [openhab.event.ItemStateUpdatedEvent ] - Item 'OG_Bad_Temperatursensor_Batterieladung' updated to 68
2023-07-21 21:01:49.617 [INFO ] [openhab.event.ItemStateUpdatedEvent ] - Item 'OG_Bad_Temperatursensor_Batterieladung' updated to 68
2023-07-21 21:01:49.617 [INFO ] [openhab.event.ItemStateUpdatedEvent ] - Item 'OG_Bad_Temperatursensor_Batterieladung' updated to 68
2023-07-21 21:01:49.617 [INFO ] [openhab.event.ItemStateUpdatedEvent ] - Item 'OG_Bad_Temperatursensor_Batterieladung' updated to 68
2023-07-21 21:01:49.617 [INFO ] [openhab.event.ItemStateUpdatedEvent ] - Item 'OG_Bad_Temperatursensor_Batterieladung' updated to 68
2023-07-21 21:01:49.617 [INFO ] [openhab.event.ItemStateUpdatedEvent ] - Item 'OG_Bad_Temperatursensor_Batterieladung' updated to 68
2023-07-21 21:01:49.624 [INFO ] [openhab.event.ItemStateUpdatedEvent ] - Item 'OG_Bad_Temperatursensor_Batterieladung' updated to 68
2023-07-21 21:01:49.646 [INFO ] [openhab.event.ItemStateUpdatedEvent ] - Item 'OG_Bad_Temperatursensor_Batterieladung' updated to 68
2023-07-21 21:01:49.646 [INFO ] [openhab.event.ItemStateUpdatedEvent ] - Item 'OG_Bad_Temperatursensor_Batterieladung' updated to 68
2023-07-21 21:01:49.647 [INFO ] [openhab.event.ItemStateUpdatedEvent ] - Item 'OG_Bad_Temperatursensor_Batterieladung' updated to 68

Currently I am on v4.0.0.M5 running in a docker. Most of my ~200 items are connected via MQTT (mosquitto, Zigbee2mqtt)

Besides the duplicates, I don’t see any problematic log entries until the memory has completely filled up, (which it did after restarting the Container after about a minute)

Is such a problem already known?

not a milestone issue but a question of how you setup your system
so post was off-topic which is why I moved it

PS yes I also had to increase -Xmx but again not a milestone issue

You should disable logging for the update event on the karat console log:set WARN openhab.event.ItemStateUpdatedEvent. It seems that on existing installations this is not always properly set when updating openHAB.

1 Like

Thanks Markus for creating this new thread and sorry for spamming the OH4 thread - that was not my intention.
Did 2024 MB work for your configuration or is it advisable to increase this value again?

Depends on bundles you use and even more on what system you talk about x86 or ARM, 32 or 64bit
You didn’t tell about yours.

On Raspi, the move to Java 17 increased my process size by 30%, I’d guess on other systems you’ll see a similar factor.

I now increased from 1024 MB to 3072 MB.
OH4 takes these 3 GB within 3 minutes without having anything to tell in the log files
(besides the openhab.event.ItemStateChangedEvent and openhab.event.GroupStateUpdatedEvent which are still at INFO level for now).
In the end, this leads to messages like:

2023-07-21 22:45:21.058 [WARN ] [ab.core.internal.events.EventHandler] - Dispatching event to subscriber 'org.openhab.core.thing.internal.CommunicationManager@1cafeff8' takes more than 5000ms.
2023-07-21 22:46:46.982 [WARN ] [io.openhabcloud.internal.CloudClient] - Socket.IO disconnected: ping timeout
2023-07-21 22:46:46.982 [INFO ] [io.openhabcloud.internal.CloudClient] - Disconnected from the openHAB Cloud service (UUID = d3...9c, base URL = http://localhost:8080)
2023-07-21 22:48:00.791 [WARN ] [ab.core.internal.events.EventHandler] - Dispatching event to subscriber 'org.openhab.core.thing.internal.CommunicationManager@1cafeff8' takes more than 5000ms.
2023-07-21 22:49:46.143 [WARN ] [ab.core.internal.events.EventHandler] - Dispatching event to subscriber 'org.openhab.core.internal.items.ItemUpdater@61fb897b' takes more than 5000ms.
2023-07-21 22:49:50.762 [WARN ] [ab.core.internal.events.EventHandler] - Dispatching event to subscriber 'org.openhab.core.io.rest.sse.internal.listeners.SseEventSubscriber@5a3572cd' takes more than 5000ms.
2023-07-21 22:49:50.765 [WARN ] [ab.core.internal.events.EventHandler] - Dispatching event to subscriber 'org.openhab.core.io.monitor.internal.metrics.EventCountMetric@1ab07938' takes more than 5000ms.
2023-07-21 22:49:55.012 [WARN ] [ab.core.internal.events.EventHandler] - Dispatching event to subscriber 'org.openhab.core.thing.internal.CommunicationManager@1cafeff8' takes more than 5000ms.
2023-07-21 22:50:04.148 [WARN ] [ab.core.internal.events.EventHandler] - Dispatching event to subscriber 'org.openhab.core.internal.items.ItemUpdater@61fb897b' takes more than 5000ms.

because no ressources are left. :confused:
In addition, the CPU load is also consistently at the limit.

The docker container is running on an DS220+ (Intel Celeron, 64 bit, 10 GB RAM). Almost all ressources are exclusively for the OH4 cotnainer.

Which addons do you use? Which scripting language?

1 Like

Amazon Echo Steuerung Binding (all Things disabled)
Astro Binding (no Thing present)
AVM FRITZ! Binding (all Things disabled)
HP Drucker Binding (all Things disabled)
iCalender Binding (all Things disabled)
MQTT Binding
Network Binding
Nuki Binding
OpenWeatherMap Binding (no Thing present)
Tankerkönig Binding (all Things disabled)
TR064 Binding (no Thing present)
Xiaomi Wifi devices (Mi IO) Binding (all Things disabled)

Persistence InfluxDB
Persistence RRD4J
Transformation SONPATH
Transformation Jinjja
Transformation MAP

JavaScript Scripting Nashorn

…ok, I now disabled most of my Things and the RAM is now stable at 1 GB. I will gradually enable them again to see who is eating up all the ressources.

Thanks Jan,
now it is obvious that it must be the iCalendar Binding.

Enabling the corresponding Thing leads to ramping up for RAM use.
I putted the add-on log to debug but do not spot anything for now.
but at least I now have a hint where to search. :+1:

3 Likes

Did you find anything? I’m just curios what happened there.

I did not enable the iCalender Binding again. Everytime I did it, RAM /CPU went high.
I use this Binding mainly for setting temperatures for the heating system. For now I am still in summer mode and push the problem to the future :grimacing:

You are free to do so. Just for the record: Series set up as such should not use much memory, many single do. But that must be a really large number. The other thing could be an eventfilter with large result set.
If analysing this issue further, the (pseudomized) ics-file and thing configuration (without password) is required. The description sounds like you tricked the binding into an endless loop. Please use the icalendar tag when asking, so i get a notification when you open a new topic here.

Thanks for the bump on this topic. I try it again now.

I think I have for now just one series appointment per two weeks (enduring for 6 days) setting one item ON/OFF with the BEGIN and END tag in the appointment.

I still have two calender Things filtering all the appointments by a special title and getting the begin and end of these. But I might delete these because I had trouble using it in rules. It was far more easy using the other method.

I might have found the problem :see_no_evil:
While deleting the calender Things I was not using anymore, I have seen that the items I connected to their channels where already deleted. This might have been the problem.

Additionally I just found this small typo in the log:
The calendar is currently offline as no local copy exists. It will go online as soon as a valid valid calendar is retrieved.

1 Like

Hello to all,

if it is ok for everyone I will continue to use this thread as I also have a memory problem and it would fit to the topic in my opinion.

I have upgraded from 4.0.1 Release Build to to 4.0.2 and did not change something in the OH setup.

Everything works well but then OH did not response as fast as before the upgrade.
I recoginzed now that I have a “Memory” issue see screenshot:
image

The openhab.log and event.log files are no longer updated.

When I execute something in openhabian I got the message that no space is left on the device (Raspberry PI) enough space.

$ fix_permissions /var/log/unattended-upgrades root:root 644 755
chown: changing ownership of '/var/log/unattended-upgrades': No space left on device
FAILED (unattended upgrades logdir)

I have changed the Java options but nothing changed.

EXTRA_JAVA_OPTS="-Xms1024m -Xmx2048m"

I have cleaned the cache:

sudo /bin/systemctl stop openhab.service
sudo openhab-cli clean-cache
sudo /bin/systemctl start openhab.service

Could it be, that there are some files, that were not deleted after the update on my PI?

Thank you for your help

Köchi