OH runs out of memory

I cannot comment on your old setup other than that openHABian is known to be a stable basis
serving many happy openHAB users. Also, I haven’t seen any post of yours on this so your motivation and attempts at analyzing and fixing your issues with it (if broken at all) can’t have been all that comprehensive.

Memory usage isn’t 800 MB but variable and a complex topic of its own.
And openHABian even has a menu offering to switch versions.
Sorry for being frank, but your knowledge on this looks too superficial to me than to make trying to roll your own system look like a good idea. This is why I recommended to go with openHABian.
Good luck if you still think it’s the better way to pursue.

1 Like

Ok, I guess there’s not much for me to achieve here.
Altough one last question remains: why is openHABian considered the better choice (besides the missing Docker overhead)? You sound like openHABian is for beginners and Docker for experts, but I cannot see why.

openHABian is the right choice for everyone including expert users.
It offers a comprehensive system setup to contain everything you need to jumpstart your installation, provides many features, it’s tested and in widespread use.
Going Docker means you have to implement (and debug) everything yourself. That approach is mostly of benefit only to some developers.

1 Like

Hi,
I took some time and moved OH back to a designated host, Raspberry Pi 4 (2 GB RAM) with openHABian. While I did not notice any logs containing a “Java heap space” message, I can see that the service restarts on its own without any announcement. This is better because at the end the service is up most of the time, but still quite annoying to have it unavailable for a certain time at random.

I would love to receive a hint on how to debug this in order to see what causes this weird behavior.

Do the log files provide some information?
It might be possible that Frontail doesn’t display enough log lines to see something there, so consider to download the openhab.log and look through it.
I don’t know how experienced you are, theoretically you could attach something like VisualVM to check for heap usage etc.

Last lines of the old log:

2022-12-27 20:41:58.813 [INFO ] [ort.loader.AbstractScriptFileWatcher] - Loading script '/etc/openhab/automation/js/environment.js'
2022-12-27 20:42:15.427 [INFO ] [.openhab.automation.openhab-js.rules] - Adding rule: Environment: Set dark outside (event)
2022-12-27 20:42:15.442 [INFO ] [.openhab.automation.openhab-js.rules] - Adding rule: Environment: Set dark outside / Night (restart)
2022-12-27 20:42:15.454 [INFO ] [.openhab.automation.openhab-js.rules] - Adding rule: Environment: Set not dark outside (event)
2022-12-27 20:42:15.464 [INFO ] [.openhab.automation.openhab-js.rules] - Adding rule: Environment: Set night
2022-12-27 20:42:15.474 [INFO ] [.openhab.automation.openhab-js.rules] - Adding rule: Environment: Set day
2022-12-27 21:03:17.482 [INFO ] [ort.loader.AbstractScriptFileWatcher] - Loading script '/etc/openhab/automation/js/ccu3.js'

First lines of the new one:

2022-12-27 21:04:10.550 [INFO ] [.core.internal.i18n.I18nProviderImpl] - Time zone set to 'Europe/Berlin'.
2022-12-27 21:04:10.632 [INFO ] [.core.internal.i18n.I18nProviderImpl] - Location set to '51.40595735299591,6.649402924638057'.
2022-12-27 21:04:10.635 [INFO ] [.core.internal.i18n.I18nProviderImpl] - Locale set to 'de_DE'.
2022-12-27 21:04:28.344 [INFO ] [el.core.internal.ModelRepositoryImpl] - Loading model 'general.test.items'
2022-12-27 21:04:28.997 [INFO ] [el.core.internal.ModelRepositoryImpl] - Loading model 'remotes.items'
2022-12-27 21:04:29.125 [INFO ] [el.core.internal.ModelRepositoryImpl] - Loading model 'room.garage.items'
2022-12-27 21:04:29.164 [INFO ] [el.core.internal.ModelRepositoryImpl] - Loading model 'radio.items'
2022-12-27 21:04:29.210 [INFO ] [el.core.internal.ModelRepositoryImpl] - Loading model 'motion.items'
2022-12-27 21:04:29.444 [INFO ] [el.core.internal.ModelRepositoryImpl] - Loading model 'environment.items'
2022-12-27 21:04:29.508 [INFO ] [el.core.internal.ModelRepositoryImpl] - Loading model 'network.items'
2022-12-27 21:04:29.540 [INFO ] [el.core.internal.ModelRepositoryImpl] - Loading model 'googleAssistant.items'
2022-12-27 21:04:29.719 [INFO ] [el.core.internal.ModelRepositoryImpl] - Loading model 'mycroft.items'
2022-12-27 21:04:29.806 [INFO ] [el.core.internal.ModelRepositoryImpl] - Loading model 'ccu3.items'

As you can see there is nothing indicating the restart. I just noticed that the tail (not frontail but actual tail -f command) stopped printing new lines.

Unfortunately I do not have any deeper knowledge when it comes to Java, but I will have a look at VIsualVM.

What is in your ccu3.js??

I removed the ccu3.js file, but the issue persist.

-rw-r--r-- 1 openhab openhab     32K Dec 28 11:47 openhab.log
-rw-r--r-- 1 openhab openhab    5.4K Dec 28 09:53 openhab.log.1.gz
-rw-r--r-- 1 openhab openhab    6.8K Dec 28 10:10 openhab.log.2.gz
-rw-r--r-- 1 openhab openhab    5.5K Dec 28 10:24 openhab.log.3.gz
-rw-r--r-- 1 openhab openhab    5.7K Dec 28 10:42 openhab.log.4.gz
-rw-r--r-- 1 openhab openhab    5.8K Dec 28 11:01 openhab.log.5.gz
-rw-r--r-- 1 openhab openhab    7.6K Dec 28 11:21 openhab.log.6.gz
-rw-r--r-- 1 openhab openhab    5.0K Dec 28 11:42 openhab.log.7.gz

You can see it restarts multiple times per hour now and creates a new log file.
Only thing I do is moving .rules to .js. Sure there are some errors in the log now and then, but even if not, it still restarts.

I am starting to think it is more related to HTTP Binding, because I very often see:

2022-12-28 13:01:27.980 [WARN ] [p.internal.http.HttpResponseListener] - Requesting 'http://192.168.178.29:81/admin/api.php' (method='GET', content='null') failed: java.util.concurrent.TimeoutException: Total timeout 3000 ms elapsed
2022-12-28 13:01:45.605 [WARN ] [p.internal.http.HttpResponseListener] - Requesting 'http://pizero/api/<XXX>/lights/6' (method='GET', content='null') failed: java.util.concurrent.TimeoutException: Total timeout 3000 ms elapsed
2022-12-28 13:01:46.624 [WARN ] [ab.core.internal.events.EventHandler] - Dispatching event to subscriber 'org.openhab.core.internal.items.ItemUpdater@8896c4' takes more than 5000ms.
2022-12-28 13:01:59.602 [WARN ] [p.internal.http.HttpResponseListener] - Requesting 'http://pizero/api/<XXX>/lights/8' (method='GET', content='null') failed: java.util.concurrent.TimeoutException: Total timeout 3000 ms elapsed
2022-12-28 13:02:10.589 [WARN ] [ab.core.internal.events.EventHandler] - Dispatching event to subscriber 'org.openhab.core.io.monitor.internal.metrics.EventCountMetric@8958fe' takes more than 5000ms.
2022-12-28 13:02:12.744 [WARN ] [p.internal.http.HttpResponseListener] - Requesting 'http://pizero/api/<XXX>/lights/5' (method='GET', content='null') failed: java.util.concurrent.TimeoutException: Total timeout 3000 ms elapsed
2022-12-28 13:02:28.848 [WARN ] [p.internal.http.HttpResponseListener] - Requesting 'http://pizero/api/<XXX>/lights/10' (method='GET', content='null') failed: java.util.concurrent.TimeoutException: Total timeout 3000 ms elapsed
2022-12-28 13:02:32.123 [WARN ] [ab.core.internal.events.EventHandler] - Dispatching event to subscriber 'org.openhab.core.internal.items.ItemUpdater@8896c4' takes more than 5000ms.

Quick test would be to disable the http binding temporarily to see if it continues. Also, I see in your logs that it is ending on scripts what are you calling in those scripts? Are you saving to any databases?

I will try.
Meanwhile I found the following in /etc/default/openhab:

EXTRA_JAVA_OPTS="-Xms192m -Xmx320m -XX:+ExitOnOutOfMemoryError"

Isn’t that drastically low? Since I am now having a dedicated host I should be able to allocate most of the 2 GB available, or not? Why is the default that low?

With -XmX320m (means heap max 320 MB) I am not wondering that it is unstable.
It requires some amount of heap.

I increased it to reasonable values and at least for the past hours everything works fine.
But the question remains why these low values were present at all. This was a fresh openHABian installation from a few days ago.

1 Like

Of what version/image ? Done on what hardware ?
Current openHABian install code will only add -mx320m if the machine you install on has less than 1.5GB.

Out of interest: Is there any reason to have not a little bit more?

Yes: memory shortage will make systems with that little RAM hang rather than restart quickly. It’s already more than the default.

1 Like

Hi Marinus,

Did you find a way to solve the memory leak?
I have the same with the docker and started when I moved from OH2 to OH3. I used multiple version and the current version is 3.4.0.RC1.

You can explicitly set the mx limit to a higher value or remove it then it’ll use the default which depends on RAM (I think for a 2GB ARM machine it’s 512m).

Beware, however, that I haven’t yet seen a system that really needs that much heap.
So if you keep hitting a 320m or even higher limit, it’s likely you have a mem leak.
Increasing the limit further will then buy you some more time but it is not a solution in that case.

1 Like

Hi @HenMus,

no, I stopped trying and went back to a dedicated Raspi for my OH. Currently waiting for a good deal for better hardware in order to move back to Docker for OH.

@mstormi thank you for the info. But if the low value of 320 is enough, why was the system running out of memory almost instantly after starting?

It is no memory leak, it’s just pretty memory consuming. My system uses about 420 MB of heap.

Or you use JS Scripting (GraalJS)…

I am currently trying out to make JS Scripting less heap hungry and faster.