Openhab 4.0.x rules need a lot of memory

No we talk from the same. That isn’t a problem with the pi itself and with memory and ram… Something is eating the memory. :slight_smile:

This memory leak is created by Openhab. So i fully agree with you, that there is something wrong.

But why is Telnet working without problems? Why is htop working?

I agree

I disagree.
I have shown a couple of times that YOUR openhab requires the same amount of memory than my system.

EDIT: Can you give us a hint why you think it is openhab, eating up your memory?

If I’m stopping openhab swap is down. That is what why i don’t understand why Openhab isn’t the issue. Sorry. Than I’m to stupid to understand this. As i show you details above what memory and swap is fully ok and it starts to rising if i start Openhab.

This is not an indication, openHAB is causing swap to grow.
It is just that you now have enough free mem for the rest.

I found a tool that show me, hope so, what process is using the swap…

pi@openhabian:~ $ sudo smem --sort=swap
  PID User     Command                         Swap      USS      PSS      RSS 
13682 ?        ?                                  0     2344     2631     4740 
13683 root     sudo smem --sort=swap              0     1900     2201     4248 
13684 root     /usr/bin/python3 /usr/bin/s        0    10392    10604    12420 
  459 root     /usr/sbin/rngd -r /dev/hwrn       48       92      101      644 
 5060 pi       sh -c node index.js               60        4       10      420 
27519 frontail tail -n 200 -F /var/log/ope       64       80      116     1224 
  382 root     /usr/sbin/cron -f                136       96      104      652 
  418 nobody   /usr/sbin/thd --triggers /e      172       28       35      532 
  381 avahi    avahi-daemon: running [open      200      408      462     1200 
27137 pi       htop                             204     2364     2575     4388 
  383 messagebus /usr/bin/dbus-daemon --syst      216      964     1027     1792 
  393 avahi    avahi-daemon: chroot helper      248        4        9      348 
  397 root     /usr/libexec/polkitd --no-d      336      392      571     1696 
24939 pi       -bash                            384     1072     1118     2496 
  423 root     /sbin/wpa_supplicant -u -s       392      956     1034     1832 
  139 root     /lib/systemd/systemd-journa      408     8988    10631    13416 
  414 root     /lib/systemd/systemd-logind      448      396      725     2700 
  343 systemd-timesync /lib/systemd/systemd-timesy      456      136      363     1528 
 1011 pi       htop -u openhab                  472     3016     3152     3860 
    1 root     /sbin/init                       548     2948     3316     5092 
  685 root     /bin/login -f                    576        4       12      544 
  562 root     sshd: /usr/sbin/sshd -D [li      652        4       36      716 
24637 pi       sshd: pi@pts/0                   740      244      336     1752 
24938 pi       sshd: pi@pts/1                   792      196      261     1496 
  430 root     /usr/libexec/bluetooth/blue      840        4       25      644 
24622 root     sshd: pi [priv]                  900        4      142     2264 
  384 root     /usr/sbin/NetworkManager --      916     2980     3271     4580 
  930 pi       /lib/systemd/systemd --user      920        4       71      860 
  407 root     /usr/sbin/rsyslogd -n -iNON      924      692      700     1228 
24931 root     sshd: pi [priv]                  932        4      126     2232 
  949 pi       -bash                            972        4       16      524 
24638 pi       -bash                            972        4       53     1496 
 1047 mosquitto /usr/sbin/mosquitto -c /etc     1240     1936     2070     2816 
  490 root     /usr/sbin/ModemManager          1348        4      173     1240 
  163 root     /lib/systemd/systemd-udevd      1452      132      139      640 
22460 root     /usr/libexec/packagekitd        1588     1548     1832     3176 
  678 root     /usr/sbin/nmbd --foreground     1796      388      490     1692 
  627 root     /usr/sbin/smbd --foreground     1984      612     1616     7676 
 1542 root     /usr/sbin/smbd --foreground     2032      668     1381     6628 
 1527 root     /usr/sbin/smbd --foreground     2068      420     1302     7208 
  616 root     /usr/sbin/smbd --foreground     2124      292      489     4268 
  931 pi       (sd-pam)                        2140        4       41      584 
 1566 root     /usr/sbin/smbd --foreground     2196      372      696     4924 
  876 root     /usr/sbin/smbd --foreground     2448       52      105     1032 
  879 root     /usr/sbin/smbd --foreground     2468       36       75      936 
  847 root     /usr/sbin/smbd --foreground     2676      180      336     1644 
  890 root     /usr/sbin/smbd --foreground     2688      312      400     1360 
  541 root     /usr/bin/python3 /usr/share     5368        4       33      740 
 8285 www-data /usr/sbin/apache2 -k start      5448     6672     7595    14784 
 8438 www-data /usr/sbin/apache2 -k start      6108      432     1030     7252 
 8357 www-data /usr/sbin/apache2 -k start      6112      428     1026     7248 
 8408 www-data /usr/sbin/apache2 -k start      6112      424     1022     7244 
 7905 www-data /usr/sbin/apache2 -k start      6116      416     1007     7176 
 8572 www-data /usr/sbin/apache2 -k start      6120      396      987     7156 
 8499 www-data /usr/sbin/apache2 -k start      6128      408     1006     7228 
31262 www-data /usr/sbin/apache2 -k start      6180      396      931     6368 
31300 www-data /usr/sbin/apache2 -k start      6188      388      924     6428 
 8599 www-data /usr/sbin/apache2 -k start      6668      132      252     2348 
26437 root     /usr/sbin/apache2 -k start      6808       64      104      996 
27398 frontail node /usr/lib/node_modules/     7732     5060     7776    12808 
  680 influxdb /usr/bin/influxd -config /e    11896    33256    33256    33264 
 5035 pi       npm start                      12432        4      958     3420 
  971 grafana  /usr/share/grafana/bin/graf    17476    40568    40579    41040 
 5061 pi       node index.js                  26904    60000    62690    67072 
27391 openhab  /usr/bin/java -XX:-UsePerfD   335004   492032   492175   493988

If i get this high swap, i will investigate what that comes from. Are you agree with me? :slight_smile:

You are welcome to do so. I don’t say this often, but you’d be completely wrong but there is no law against that.

Yes you do! It’s taking so long to swap stuff in and out of RAM that safety measures built into OH to prevent a thread from running forever kick in and start killing things and generating warnings and errors.

The system load in your htop screen shots indicate otherwise. A system load above 1 is usually considered a problem. One of your load averages is 6 :scream: .

It’s not just openHAB. Swap is a system level resource. The sum total of all your processes RAM requirements is greater than the RPi 3 can handle. You are just noticing the problem with OH because you are using it, and because many parts of OH are used relatively infrequently, OH’s RAM is the most likely to be swapped out in the first place.

Based on your htop screen shots, OH isn’t really using an unusual amount of RAM. Maybe you can address this system problem by changes to OH, but OH is not the problem. You have too much stuff running on this machine.

OH has a set limit in the total amount of memory it can consume. Once it hits that limit it crashes and exits. OH does not have a memory leak here or else you’d see OutOfMemory exceptions in openhab.log

They are probably taking a little bit longer to start up than usual. But once started, there RAM is constantly being used so unlikely to be swapped out. But that means stuff that isn’t being used constantly (e.g. parts of OH memory) is being swapped out.

Total RAM usage is the problem. Of course memory use and swap use is going to go down when you stop OH. OH uses a lot of RAM. What happens if you leave OH running and stop InfluxDB and Grafana? Does the swap use go down? Does the memory use go down? Of course it does because both of those also use RAM and, if RAM runs out swap.

No because the programs themselves do not get to decide when their memory goes to swap (yes, I know there are ways you can pin memory so it doesn’t get swapped but to my knowledge OH doesn’t use that and I’m not certain Java even supports that). All that will show you is those programs have more stuff in memory that isn’t used as frequently as stuff in memory used by other programs.

Ultimately most of this thread is coming from confirmation bias. You are convinced that OH is the problem so much so that you’ve not even considered looking for any other evidence and in fact have rejected evidence that could point to something else, even though we’ve explained at length the reasons why OH is not the root of the problem, running too much on the machine is.

tl;dr: Based on your own screen shots:

  • OH is not using an unusual amount of RAM on your machine
  • there are no OOM exceptions reported so there is no memory leak
  • all of your RAM and a good chunk of your swam are in use
  • all of the problems with OH reported here can be explained by the fact that swap is in use

I’ve presented a full analysis and explanation as has @Oliver2 . I believe other experts on this thread agree with these conclusions. You are free to disagree or reject that analysis but, thus far, all of your arguments against it denote a misunderstanding of how computers work (that’s not a dig, this isn’t something most people ever have to know about or deal with). I’ve tried but I apparently can’t fix misunderstanding.

I wish you the best of luck! I can offer nothing else here.

@rlkoshak thank you very much for your fully explanation and your help.

At the end I found a program that show me which process needs swap and i will share it, if the issue comes up again.

Yes, because openhab frees up 400MB of your memory

This seems me to be where your misunderstanding lies. In terms of system performance any swap is “high” swap. Yes, you have more drive space allocated to hold swapped memory but that doesn’t mean anything at all (and can be arbitrarily configured). You’ve already hit you peak RAM if the system has had to move anything at all to the swap space. Swap is an emergency stopgap, not a long term solution for system performance. If a system is consistently running with swap memory in use than that system is not properly matched between its hardware and usage profile. Period. End of story.

Instead of looking at what is using the swap memory, you have investigate why the system needs swap memory in the first place, and this has already been answered.

An RPI3 is barely enough to run OH4 without any additional load. You are adding additional load.
The story doesn’t go beyond this. There isn’t a memory leak. It doesn’t matter at all which load is which. It simply adds up to more than can be held in RAM and the system must resort to swap.

That’s a perfectly reasonable test. Now test what happens if you just run openhab without all the other services on the system. What happens then? I can tell you if you don’t want to check yourself: the swap usage will also be down.

Openhab is only the issue you see. I promise you that when your openhab rules start running all the other services have issues as well. Does your rule change an item state? Well then that item is persisted to you InfluxDB, but maybe that was swapped out for the rule thread and now the system has to swap back. Does Grafana connect to InfluxDB (don’t know why you’d have it if it doesn’t), now the Grafana update is also impacted…etc.

Quite frankly, with everything you’re trying to run, I’m surprised your rules only take 5 to 10 minutes.

1 Like

No… Openhab isn’t response itself. Not the rules take this time… I must wait 5-10min and everything is working. All rules items aren’t working in this time.

Interesting…
Could you do a

smem -t -p

please?

Right, you must wait while the system shuffles around RAM and swap memory to get even the basic OH system processes running.

Not according to your original post where openhan will then crash a short time later. That crash is because the swap issue continues to build a back log until you do finally completely run out of system resources.

  PID User     Command                         Swap      USS      PSS      RSS 
  393 avahi    avahi-daemon: chroot helper    0.01%    0.00%    0.00%    0.03% 
  685 root     /bin/login -f                  0.03%    0.00%    0.00%    0.05% 
 5060 pi       sh -c node index.js            0.00%    0.00%    0.00%    0.04% 
  949 pi       -bash                          0.05%    0.00%    0.00%    0.05% 
  430 root     /usr/libexec/bluetooth/blue    0.04%    0.00%    0.00%    0.06% 
  541 root     /usr/bin/python3 /usr/share    0.28%    0.00%    0.00%    0.07% 
  418 nobody   /usr/sbin/thd --triggers /e    0.01%    0.00%    0.00%    0.05% 
  562 root     sshd: /usr/sbin/sshd -D [li    0.03%    0.00%    0.00%    0.07% 
  931 pi       (sd-pam)                       0.11%    0.00%    0.00%    0.06% 
26437 root     /usr/sbin/apache2 -k start     0.38%    0.00%    0.00%    0.07% 
24638 pi       -bash                          0.05%    0.00%    0.01%    0.15% 
 8599 www-data /usr/sbin/apache2 -k start     0.38%    0.00%    0.01%    0.15% 
31262 www-data /usr/sbin/apache2 -k start     0.39%    0.00%    0.01%    0.14% 
  879 root     /usr/sbin/smbd --foreground    0.14%    0.00%    0.01%    0.09% 
  930 pi       /lib/systemd/systemd --user    0.05%    0.00%    0.01%    0.09% 
31300 www-data /usr/sbin/apache2 -k start     0.39%    0.00%    0.01%    0.15% 
  876 root     /usr/sbin/smbd --foreground    0.14%    0.00%    0.01%    0.09% 
 7905 www-data /usr/sbin/apache2 -k start     0.39%    0.00%    0.01%    0.19% 
 8357 www-data /usr/sbin/apache2 -k start     0.39%    0.00%    0.01%    0.19% 
 8408 www-data /usr/sbin/apache2 -k start     0.39%    0.00%    0.01%    0.19% 
 8438 www-data /usr/sbin/apache2 -k start     0.39%    0.00%    0.01%    0.19% 
 8499 www-data /usr/sbin/apache2 -k start     0.39%    0.00%    0.01%    0.19% 
 8572 www-data /usr/sbin/apache2 -k start     0.39%    0.00%    0.01%    0.19% 
  459 root     /usr/sbin/rngd -r /dev/hwrn    0.00%    0.01%    0.01%    0.06% 
  382 root     /usr/sbin/cron -f              0.01%    0.01%    0.01%    0.07% 
27519 frontail tail -n 200 -F /var/log/ope    0.00%    0.01%    0.01%    0.12% 
24931 root     sshd: pi [priv]                0.05%    0.00%    0.01%    0.22% 
24622 root     sshd: pi [priv]                0.05%    0.00%    0.01%    0.23% 
  163 root     /lib/systemd/systemd-udevd     0.08%    0.01%    0.01%    0.06% 
  490 root     /usr/sbin/ModemManager         0.07%    0.00%    0.02%    0.12% 
24637 pi       sshd: pi@pts/0                 0.04%    0.02%    0.03%    0.17% 
24938 pi       sshd: pi@pts/1                 0.04%    0.02%    0.03%    0.15% 
 8285 www-data /usr/sbin/apache2 -k start     0.48%    0.00%    0.03%    0.28% 
  847 root     /usr/sbin/smbd --foreground    0.15%    0.02%    0.03%    0.15% 
  890 root     /usr/sbin/smbd --foreground    0.15%    0.03%    0.04%    0.12% 
  343 systemd-timesync /lib/systemd/systemd-timesy    0.02%    0.01%    0.04%    0.15% 
  616 root     /usr/sbin/smbd --foreground    0.12%    0.03%    0.05%    0.42% 
  678 root     /usr/sbin/nmbd --foreground    0.10%    0.04%    0.05%    0.16% 
  381 avahi    avahi-daemon: running [open    0.01%    0.04%    0.05%    0.12% 
 1527 root     /usr/sbin/smbd --foreground    0.13%    0.03%    0.05%    0.43% 
 1542 root     /usr/sbin/smbd --foreground    0.13%    0.03%    0.05%    0.43% 
  627 root     /usr/sbin/smbd --foreground    0.13%    0.03%    0.05%    0.44% 
 1566 root     /usr/sbin/smbd --foreground    0.13%    0.03%    0.05%    0.44% 
  397 root     /usr/libexec/polkitd --no-d    0.02%    0.04%    0.06%    0.17% 
  414 root     /lib/systemd/systemd-logind    0.02%    0.03%    0.06%    0.26% 
  407 root     /usr/sbin/rsyslogd -n -iNON    0.05%    0.07%    0.07%    0.12% 
  383 messagebus /usr/bin/dbus-daemon --syst    0.01%    0.09%    0.09%    0.17% 
 5035 pi       npm start                      0.66%    0.00%    0.10%    0.34% 
  423 root     /sbin/wpa_supplicant -u -s     0.02%    0.10%    0.10%    0.18% 
24939 pi       -bash                          0.02%    0.11%    0.11%    0.25% 
22460 root     /usr/libexec/packagekitd       0.08%    0.12%    0.14%    0.28% 
18161 root     sudo smem -t -p                0.00%    0.19%    0.22%    0.42% 
 1047 mosquitto /usr/sbin/mosquitto -c /etc    0.06%    0.23%    0.23%    0.30% 
27137 pi       htop                           0.01%    0.24%    0.26%    0.44% 
    1 root     /sbin/init                     0.03%    0.23%    0.27%    0.44% 
 1011 pi       htop -u openhab                0.03%    0.30%    0.32%    0.39% 
  384 root     /usr/sbin/NetworkManager --    0.05%    0.30%    0.32%    0.45% 
27398 frontail node /usr/lib/node_modules/    0.46%    0.43%    0.71%    1.21% 
18162 root     /usr/bin/python3 /usr/bin/s    0.00%    1.06%    1.07%    1.25% 
  139 root     /lib/systemd/systemd-journa    0.02%    1.44%    1.64%    1.95% 
  971 grafana  /usr/share/grafana/bin/graf    0.90%    4.28%    4.28%    4.32% 
  680 influxdb /usr/bin/influxd -config /e    0.43%    4.72%    4.72%    4.72% 
 5061 pi       node index.js                  1.27%    6.68%    6.96%    7.40% 
27391 openhab  /usr/bin/java -XX:-UsePerfD   19.14%   48.79%   48.80%   48.98% 
-------------------------------------------------------------------------------
   64 12                                     29.99%   69.85%   71.25%   80.97%

These are smaller programs which might even fit into the remaining 15% memory space or where you do not really notice if the system is engaged a few seconds with swapping

Sorry, for a consistent view we need these information from the same time. Just to avoid that recently parts of oh have been swapped.

  • htop screenshot from the top of the screen and where we can see the memory usage of OH (or VM thread :slight_smile: )
  • swapon -s
  • smem -t -p

I have a question as I haven’t found information about this.
Is your system a migrated installation from Buster to Bullseye ?
In case it is was that a 32 bit Buster installation ?
In case it was what is the output of command: uname -a after this migration / now ?

Its Bullseye and yes its migrated from Buster to Bullseye.

Linux openhabian 6.1.43-v7+ #1669 SMP Thu Aug  3 16:23:38 BST 2023 armv7l GNU/Linux
No LSB modules are available.
Distributor ID:	Raspbian
Description:	Raspbian GNU/Linux 11 (bullseye)
Release:	11
Codename:	bullseye
1 Like

This can cause issues as well, so would be worth to try with a clean bullseye installation.

Sure that is a good point. Now i will wait what we get with the commands for swap. Before I start with a fresh install… And all guys here gave me so many hints. I will post this here and then I can decide what I do as next. But at the moment swap isn’t high to collect information’s.

I think you have a few bindings installed in the addons directory. Did you make sure with their maintainers that all of them are working with OH4?
See here