Openhab 4.0.x rules need a lot of memory

Your log shows you the symptoms of your problem, not the root cause.
As you saw in the other thread where other users had a problem with CPU being at 100%, it was not obvious to anyone in the beginning that the problem was a bug in JDK17 (which needs to be installed for OH4).

Maybe your problem has a similiar reason which we (including YOURSELF) need to find out. Maybe there is no problem and we might find out due to the changes of OH4 and the new system requirements (mainly JDK17 and buster) that a PI with 4GB is the new minimum requirement especially when using MQTT and Grafana. Who knows - we need to find out.

The majority of work in the beginning is on your side whereas the experts have the majority of work towards the end (bugfixing). What I want to say is that everybody has to contribute a little bit of work especially when there are problems with a new version.

You already got a few recommendations from experts in this thread above. Please go ahead and try to narrow it down so that the experts here have a starting point.

You said you do not have time to start analysing YOUR problem? But you already had time to write posts about your problem the last 7 days… (no offense - just realising)

1 Like

I just like to add my memory consumption which is almost the same than your system:

16.2% of 4GB =~600MB
Your system: 56% of 1GB = ~600MB
At least we can say that you and me have no problem or we have the same problem :slight_smile:

Would be interesting to know how much memory OH3.4 consumed.

1 Like

I respect everyone spend time to develop and I respect everyone how try to help me for fixing and I’m really glad that Openhab exist! That isn’t the core of all. You just see me here not often, but via Facebook I help to try many guys there if I can in the German Group. If you don’t see me here in the first row, that doesn’t mean that I’m a parasite who only criticism the system! But if you tell me, that I sit here to switch all rules off (90 rules) and Bindings (Dunno 15 but 120+ things) step by step for an event that happens not often but it does twice or more, than sorry I’m out! How many days I’m need for this? Month? Meanwhile OH4.1 is out…

I don’t say that’s your problem, i say exactly that’s my problem and I can only hope that someone else get in trouble, too. If i decide that this is riding the horse in hell and it don’t make sens to me to invest more time in something that nobody can do! Also you must respect me.

BTW OH3 and OH4 looks very similar with RAM consumption… If I remember right I was also round about 700mb… But with OH3 I had a ZWAVE system with ZWAVE.me solution also running and everything was OK. OH3 runs also with Z2M, ZWAVE, Grafana, Influx and an apache2 webserver… Everything was nice. But now i get in trouble with OH4. In the meanwhile i decided to kill ZWAVE and switch over to z2m completely. With OH4 there is more RAM free cause I don’t use ZWAVE anymore.

Ah, yeah an MQTT Broker for sure is running also…

All good Swen. You misunderstood - I am not complaining that you are not contributing in general. I am not complaining at all :slight_smile:

I described one of potential approaches. But I would be interested to see how your system works if you disabled all rules which can be done easily through MainUI in one minute. Do you have access to MainUI at least shortly after a restart?

When upgrading from 3.4 to 4 did you follow the upgrade instructions so that the upgrade script was able to be run?

Thanks for your help and understanding. :slight_smile:

Yes sure. After update to 4.0.2-1 it looks more stable as before. But it would be nice to find the hick up also when I remove bindings and reinstall them. The DenonMarantz problem is yet fixed also what I get via SYSlog, but Openhab do the same issue after time.

No Openhab is running:

Openhab starts:

2023-08-18 15:48:45.958 [WARN ] [core.karaf.internal.FeatureInstaller] - The transformation add-on 'javascript' does not exist - ignoring it.

Openhab without Rule engine started:

Openhab with Rule engine started:

How it is when it’s normal you can see above.

The UI is working after start without issue and running very fast.

Don‘t know if this had been asked before, but did you update your rules to use GraalVM and not Nashorn?
Also check the config for 3.x leftovers…
This one might be helpfull

Yes, sure that is next point what i do after upgrade. I had to fix all rules. And the system crash all over the time and rules are sooooo slow if they are started in the first time. Also And I thing, hey if i must touch all rules, I can convert them directly to get them up to date with the new GraalVM. The log is now cleare without issues of any rule hope for sure. I never comes over 24h cause of fix or try something with restart. :slight_smile:

What are leftovers in config from 3.x? Did you have more details? Must I delete something?

Other Thread I will check. But all things are created by UI.

As Rich was already mentioning the swap file is quite huge. My total mem consumption is 740 MB. Your is 1.4GB.
Can you do a htop screenshot sorted by memory (without threads) and scroll down for a while until the next non-oh related command is shown, like this:

What is the output of

sudo service dphys-swapfile status.

could be that addons.config in userdata/config (not at home, therefore cannot validate path), might still refer to no longer existing addons.

If you don’t want to spend a lot of time on this I think your options are clear.

  1. Move stuff off of the RPi3 (e.g. InfluxDB and Grafana) to give OH more RAM to work with
  2. Abandon InfluxDB and Grafana for something that requires less memory (e.g. rrd4j and MainUI charts)
  3. Move everything to a machine with more RAM

Given the symptoms, any one of these should fix your problem.

Anything else is going to take a lot of time and effort and you still may end up with the same three options when you are done.

There may be some half measures you can use for the three options above. For example, maybe just close down Grafana and use MainUI charting. Research InfluxDB to see if you can configure it to use less RAM.

But as far as I can tell, the problem here is that everything you are running on this machine needs about 1.4-1.7 GB of RAM and you only have 1 GB of RAM. The only way to fix it is going to be reducing the amount of RAM required to 1 GB which is a huge reduction. I doubt you’ll ever be able to reach that through tweaks to configs and command line arguments.

Oh I’ve got that one memorized. ;-). $OH_USERDATA/config/org/openhab/addons.config

1 Like
● dphys-swapfile.service - dphys-swapfile - set up, mount/unmount, and delete a swap file
     Loaded: loaded (/lib/systemd/system/dphys-swapfile.service; enabled; vendor preset: enabled)
     Active: active (exited) since Wed 2023-08-16 15:17:12 CEST; 2 days ago
       Docs: man:dphys-swapfile(8)
   Main PID: 457 (code=exited, status=0/SUCCESS)
      Tasks: 0 (limit: 2176)
        CPU: 0
     CGroup: /system.slice/dphys-swapfile.service

Warning: journal has been rotated since unit was started, output may be incomplete.
openhab  27391 96.1 62.9 1164640 626336 ?      Ssl  16:00  26:13 /usr/bin/java -XX:-UsePerfData -Dopenhab.home=/usr/share/openhab -Dopenhab.conf=/etc/openhab -Dopenhab.runtime=/usr/share/openhab/runtime -Dopenhab.userdata=/var/lib/openhab -Dopenhab.logdir=/var/log/openhab -Dfelix.cm.dir=/var/lib/openhab/config -Djava.library.path=/var/lib/openhab/tmp/lib -Djdk.util.zip.disableZip64ExtraFieldValidation=true -Djetty.host=0.0.0.0 -Djetty.http.compliance=RFC2616 -Dorg.apache.cxf.osgi.http.transport.disable=true -Dorg.ops4j.pax.web.listening.addresses=0.0.0.0 -Dorg.osgi.service.http.port=8080 -Dorg.osgi.service.http.port.secure=8443 -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Xms192m -Xmx768m -XX:+ExitOnOutOfMemoryError --add-reads=java.xml=java.logging --add-exports=java.base/org.apache.karaf.specs.locator=java.xml,ALL-UNNAMED --patch-module java.base=/usr/share/openhab/runtime/lib/endorsed/org.apache.karaf.specs.locator-4.4.3.jar --patch-module java.xml=/usr/share/openhab/runtime/lib/endorsed/org.apache.karaf.specs.java.xml-4.4.3.jar --add-opens java.base/java.security=ALL-UNNAMED --add-opens java.base/java.net=ALL-UNNAMED --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.naming/javax.naming.spi=ALL-UNNAMED --add-opens java.rmi/sun.rmi.transport.tcp=ALL-UNNAMED --add-opens java.base/java.io=ALL-UNNAMED --add-opens java.base/java.lang.reflect=ALL-UNNAMED --add-opens java.base/java.text=ALL-UNNAMED --add-opens java.base/java.time=ALL-UNNAMED --add-opens java.desktop/java.awt.font=ALL-UNNAMED --add-exports=java.base/sun.net.www.protocol.file=ALL-UNNAMED --add-exports=java.base/sun.net.www.protocol.ftp=ALL-UNNAMED --add-exports=java.base/sun.net.www.protocol.http=ALL-UNNAMED --add-exports=java.base/sun.net.www.protocol.https=ALL-UNNAMED --add-exports=java.base/sun.net.www.protocol.jar=ALL-UNNAMED --add-exports=java.base/sun.net.www.content.text=ALL-UNNAMED --add-exports=jdk.xml.dom/org.w3c.dom.html=ALL-UNNAMED --add-exports=jdk.naming.rmi/com.sun.jndi.url.rmi=ALL-UNNAMED --add-exports=java.rmi/sun.rmi.registry=ALL-UNNAMED --add-exports=java.security.sasl/com.sun.security.sasl=ALL-UNNAMED --add-exports=java.naming/com.sun.jndi.ldap=ALL-UNNAMED -Dkaraf.instances=/var/lib/openhab/tmp/instances -Dkaraf.home=/usr/share/openhab/runtime -Dkaraf.base=/var/lib/openhab -Dkaraf.data=/var/lib/openhab -Dkaraf.etc=/var/lib/openhab/etc -Dkaraf.log=/var/log/openhab -Dkaraf.restart.jvm.supported=true -Djava.io.tmpdir=/var/lib/openhab/tmp -Djava.util.logging.config.file=/var/lib/openhab/etc/java.util.logging.properties -Dkaraf.startLocalConsole=false -Dkaraf.startRemoteShell=true -classpath /usr/share/openhab/runtime/lib/boot/org.apache.karaf.diagnostic.boot-4.4.3.jar:/usr/share/openhab/runtime/lib/boot/org.apache.karaf.jaas.boot-4.4.3.jar:/usr/share/openhab/runtime/lib/boot/org.apache.karaf.main-4.4.3.jar:/usr/share/openhab/runtime/lib/boot/org.apache.karaf.specs.activator-4.4.3.jar:/usr/share/openhab/runtime/lib/boot/osgi.core-8.0.0.jar:/usr/share/openhab/runtime/lib/jdk9plus/istack-commons-runtime-3.0.10.jar:/usr/share/openhab/runtime/lib/jdk9plus/jakarta.xml.bind-api-2.3.3.jar:/usr/share/openhab/runtime/lib/jdk9plus/javax.annotation-api-1.3.2.jar:/usr/share/openhab/runtime/lib/jdk9plus/jaxb-runtime-2.3.3.jar:/usr/share/openhab/runtime/lib/jdk9plus/org.apache.servicemix.specs.activation-api-1.2.1-1.2.1_3.jar:/usr/share/openhab/runtime/lib/jdk9plus/txw2-2.3.3.jar org.apache.karaf.main.Main
pi        5061  7.8  6.9 229968 68832 ?        Sl   Aug16 213:20 node index.js
grafana    971  0.4  3.8 907516 38472 ?        Ssl  Aug16  14:13 /usr/share/grafana/bin/grafana server --config=/etc/grafana/grafana.ini --pidfile=/run/grafana/grafana-server.pid --packaging=deb cfg:default.paths.logs=/var/log/grafana cfg:default.paths.data=/var/lib/grafana cfg:default.paths.plugins=/var/lib/grafana/plugins cfg:default.paths.provisioning=/etc/grafana/provisioning
influxdb   680  0.7  3.7 1073756 37236 ?       Sl   Aug16  22:19 /usr/bin/influxd -config /etc/influxdb/influxdb.conf
root       139  1.1  1.6  92820 16284 ?        Ss   Aug16  33:16 /lib/systemd/systemd-journald
frontail 27398  0.3  1.2 122952 12796 ?        Ssl  16:00   0:05 node /usr/lib/node_modules/frontail/bin/frontail --disable-usage-stats --ui-highlight --ui-highlight-preset /usr/lib/node_modules/frontail/preset/openhab_AEM.json --theme openhab_AEM --lines 2000 --number 200 /var/log/openhab/openhab.log /var/log/openhab/events.log
root         1  0.0  0.4  34792  4764 ?        Ss   Aug16   0:37 /sbin/init
pi       27137  3.8  0.4  10500  4612 pts/0    S+   15:56   1:11 htop
openhab  32445  0.3  0.4   8568  4556 ?        S    16:27   0:00 arping -w 5 -C 1 -i enxb827eb1224b9 192.168.179.102
pi        1011  4.6  0.3  11496  3840 tty1     S+   Aug16 136:30 htop -u openhab
root       384  0.0  0.3  60728  3660 ?        Ssl  Aug16   1:41 /usr/sbin/NetworkManager --no-daemon
pi        5035  0.0  0.3 160288  3416 ?        Ssl  Aug16   0:03 npm start
pi       24939  0.0  0.3   8556  3024 pts/1    Ss   15:47   0:00 -bash
pi       32455  0.0  0.2  11000  2664 pts/1    R+   16:28   0:00 ps aux --sort=-%mem
root     24622  0.0  0.2  14436  2312 ?        Ss   15:44   0:00 sshd: pi [priv]
root     24931  0.0  0.2  14436  2292 ?        Ss   15:47   0:00 sshd: pi [priv]
root       414  0.0  0.2  21176  2196 ?        Ss   Aug16   0:01 /lib/systemd/systemd-logind
root       423  0.0  0.1  11956  1772 ?        Ss   Aug16   0:36 /sbin/wpa_supplicant -u -s -O /run/wpa_supplicant
mosquit+  1047  0.3  0.1  13552  1768 ?        Ss   Aug16  10:26 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf
root       678  0.0  0.1  70676  1716 ?        Ss   Aug16   0:45 /usr/sbin/nmbd --foreground --no-process-group
pi       24638  0.0  0.1   8556  1712 pts/0    Ss   15:44   0:00 -bash
pi       24637  0.2  0.1  14436  1668 ?        S    15:44   0:06 sshd: pi@pts/0
root       890  0.0  0.1  84512  1504 ?        S    Aug16   0:01 /usr/sbin/smbd --foreground --no-process-group
frontail 27519  0.0  0.1   6988  1488 ?        S    16:00   0:00 tail -n 200 -F /var/log/openhab/openhab.log /var/log/openhab/events.log
pi       24938  0.0  0.1  14436  1412 ?        S    15:47   0:00 sshd: pi@pts/1
root     22460  0.0  0.1  95620  1380 ?        Ssl  Aug16   0:01 /usr/libexec/packagekitd
message+   383  0.0  0.1   7828  1324 ?        Ss   Aug16   0:47 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
root       847  0.0  0.1  84524  1248 ?        Ss   Aug16   0:01 /usr/sbin/smbd --foreground --no-process-group
systemd+   343  0.0  0.1  22208  1236 ?        Ssl  Aug16   0:01 /lib/systemd/systemd-timesyncd
avahi      381  0.0  0.1   6832  1220 ?        Ss   Aug16   0:52 avahi-daemon: running [openhabian.local]
root       397  0.0  0.1  39468  1180 ?        Ssl  Aug16   0:00 /usr/libexec/polkitd --no-debug
root       407  0.5  0.1  26460  1048 ?        Ssl  Aug16  14:59 /usr/sbin/rsyslogd -n -iNONE
root       490  0.0  0.0  48496   924 ?        Ssl  Aug16   0:00 /usr/sbin/ModemManager
pi         930  0.0  0.0  14324   860 ?        Ss   Aug16   0:00 /lib/systemd/systemd --user
root       876  0.0  0.0  82716   800 ?        S    Aug16   0:00 /usr/sbin/smbd --foreground --no-process-group
root       879  0.0  0.0  82720   764 ?        S    Aug16   0:00 /usr/sbin/smbd --foreground --no-process-group
root       562  0.0  0.0  12328   712 ?        Ss   Aug16   0:00 sshd: /usr/sbin/sshd -D [listener] 0 of 10-100 startups
root     26437  0.0  0.0 217220   704 ?        Ss   Aug16   0:14 /usr/sbin/apache2 -k start
root       382  0.0  0.0   8116   652 ?        Ss   Aug16   0:00 /usr/sbin/cron -f
root       163  0.0  0.0  20004   640 ?        Ss   Aug16   0:01 /lib/systemd/systemd-udevd
root       459  0.0  0.0  27612   628 ?        SLsl Aug16   0:00 /usr/sbin/rngd -r /dev/hwrng
openhab  32444  0.0  0.0   8428   628 ?        S    16:27   0:00 ping -w 5 -c 1 192.168.179.102
www-data 13780  0.0  0.0 218204   620 ?        S    00:00   0:00 /usr/sbin/apache2 -k start
www-data 31018  0.0  0.0 218196   608 ?        S    09:14   0:00 /usr/sbin/apache2 -k start
root       430  0.0  0.0  21368   576 ?        Ss   Aug16   0:00 /usr/libexec/bluetooth/bluetoothd
www-data 30675  0.0  0.0 217300   572 ?        S    09:12   0:00 /usr/sbin/apache2 -k start
www-data 30786  0.0  0.0 217300   572 ?        S    09:13   0:00 /usr/sbin/apache2 -k start
www-data 31059  0.0  0.0 217300   572 ?        S    09:14   0:00 /usr/sbin/apache2 -k start
www-data 31242  0.0  0.0 217300   564 ?        S    09:16   0:00 /usr/sbin/apache2 -k start
www-data 31300  0.0  0.0 217252   540 ?        S    09:16   0:00 /usr/sbin/apache2 -k start
nobody     418  0.0  0.0   5228   532 ?        Ss   Aug16   0:01 /usr/sbin/thd --triggers /etc/triggerhappy/triggers.d/ --socket /run/thd.socket --user nobody --deviceglob /dev/input/event*
pi         949  0.0  0.0   8564   524 tty1     S    Aug16   0:00 -bash
root       685  0.0  0.0   8180   472 tty1     Ss   Aug16   0:00 /bin/login -f
www-data 31084  0.0  0.0 217300   460 ?        S    09:14   0:00 /usr/sbin/apache2 -k start
www-data 31132  0.0  0.0 217300   460 ?        S    09:15   0:00 /usr/sbin/apache2 -k start
root       541  0.0  0.0  40568   448 ?        Ssl  Aug16   0:00 /usr/bin/python3 /usr/share/unattended-upgrades/unattended-upgrade-shutdown --wait-for-signal
www-data 31262  0.0  0.0 217300   272 ?        S    09:16   0:00 /usr/sbin/apache2 -k start
avahi      393  0.0  0.0   6672     4 ?        S    Aug16   0:00 avahi-daemon: chroot helper
pi         931  0.0  0.0  36244     4 ?        S    Aug16   0:00 (sd-pam)

@rlkoshak I don’t understand you again. Sorry… But I see 750mb ram is in use and only 296mb swap is in use exactly in this moment. I don’t see that is something wrong here…
If openhab stops then there is 172MB in use and swap is 154MB… So tell me please whats wrong with it? I can’t find the issue at the moment. If swap will be filled totally than there is a problem with Openhab or Openhab needs 2GB of RAM. Then all users will have problems to run Openhab 4 on Pi 3. That don’t make seens to me sorry.

This all looks good. Last question, output of

swapon -s
Filename				Type		Size	Used	Priority
/var/swap                              	file    	1890300	312472	-2

BTW there is no bad entry in addons.config

:org.apache.felix.configadmin.revision:=L"141"
automation=",jsscripting"
binding="ipcamera,netatmo,dwdunwetter,tr064,pixometer,telegram,ntp,dwdpollenflug,tankerkoenig,harmonyhub,avmfritz,openweathermap,network,miio,astro,mqtt,icalendar,http,exec"
felix.fileinstall.filename="file:/var/lib/openhab/etc/org.openhab.addons.cfg"
legacy=B"false"
misc="openhabcloud"
package="standard"
persistence=",influxdb"
remote=B"true"
service.pid="org.openhab.addons"
transformation="exec,javascript,map,jsonpath,regex,jinja"
ui="basic"
voice="picotts"

According to this screenshot, 1.4 GB are in use

Yes sure something is eating the mem… But for me that is no problem. I we can discuss if swap is also full.

No, it is dedicated to OH. Persistence is rrd4j but with files stored on a mounted drive.

To run just OH4 and a standard setup (18 bindings installed in my case), a RPI 3 seems to be ok with no problem. Maybe just a little slow to start.

@Lolodomo thanks for your feedback.

So i will say, the Pi is ok with RAM and SWAP.

So nobody had explained me what VM Thread is and what that thing doing. That is on the second position if nothing works for 5-10min on htop list.

Remember to that picture:
grafik

If I roughly add up my CPU of all various commands I end up almost exactly with the 22% out of 4GB ~ 800MB which is inline with the memory bar of htop in the upper left corner (no surprise).

If I do the same with your system, I end up with ~84% out of 1GB = 840MB which is also fine, BUT swapon showed us that another 312MB are in use.
This is to me unusual.
Maybe you get some tools so that you can identify which processes are using swap file.

Yes then we end in openhab. But what i say… We talk about something what we don’t need really. We can talk about it, with the swap is full. What i say… I it doesn’t matter if swap is 400 or 800… Cause if the swap is so high openhab is running! We don’t need talk about… I don’t say Openhab isn’t running.

But there is a process depended with VM Thread what openhab blocked to working… It needs then 5-10mins and Openhab is working… And we must investigate this!