Out of memory: Killed process 364 (java) and OH3

[Simon_B (Simon Ban)]

Hey Guys,
i have moved from opehabian PI4 setup to an Proxmox VM with 6 GB Ram and since few days the OH3 restarts very often:

[  161.687828] Out of memory: Killed process 364 (java) total-vm:11203364kB, anon-rss:5556504kB, file-rss:0kB, shmem-rss:0kB, UID:107 pgtables:13816kB oom_score_adj:0

  5963]   107  5963  2761751  1399651 13869056   211538             0 java
[ 2635.678693] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/system.slice/openhab.service,task=java,pid=5963,uid=107
[ 2635.678993] Out of memory: Killed process 5963 (java) total-vm:11047004kB, anon-rss:5598836kB, file-rss:0kB, shmem-rss:0kB, UID:107 pgtables:13544kB oom_score_adj:0

I have set :
EXTRA_JAVA_OPTS=“-Xms2048m -Xmx4096m”

Could someone help me to find out what is the cuase for the reboots.

Regards
Simon

[Wolfgang_S]

‘Something’ is eating your systems memory …
The out of memory (oom) kill process identify the java process and kill it.

See background about oom killer; how to configure and disable it:

Linux Memory Overcommitment and the OOM Killer | Baeldung on Linux

https://www.oracle.com/technical-resources/articles/it-infrastructure/dev-oom-killer.html

You need to identify which process(es) eat up the memory.
For openhab this can be done by disabling binding by binding.
There are several threads about it.

[Simon_B (Simon Ban)]

Hi Wolfgang,
thanks for the link, i have checked your link and it seems that the following processess

9542 666 bash
425 667 unattended-upgr
10229 671 node
10938 680 ffmpeg
10228 858 java

Since OH3 is the only Service running i will disable the Bindings or Items and check which could cause the issue.

Update:
After disabling my Synology Binding it seems to be stable now. I will monitor it:
235 │ Active │ 80 │ 3.3.0.202207031140 │ openHAB Add-ons :: Bundles :: Synology Surveillance Station Binding

[Simon_B (Simon Ban)]

After enabling the Synology Binding the Out of memory: Killed process 13814 (java) appears and OH3 is restarting.
@Pavion could we check this issue?

Regards
Simon

[Pavion]

Hi Simon,
I’m unaware of any memory issues but I admit, memory profiling is not my field of experience. There are no libraries used which I’d suspect. If you have any debug output from this binding and / or core, please PM some.
BR
Pav

[Simon_B (Simon Ban)]

@Pavion Thx for your reply.

[Simon_B (Simon Ban)]

i need to think about it.

[hmerk (Hans-Jörg Merk)]

This is not the solution, as openHAB 3 is not confirmed to run without issues on Java versions higher than 11.

Please note that we had to make a couple of changes within ongoing openHAB 4 development to fully support Java 17.

I stongly recommend to switch back to Java 11 and further investigate on the root cause for your restarts.

You are misleading others with your last post!

[Simon_B (Simon Ban)]

ok i understand, but what can i do? I find out the Synology Binding was causing the reboots.
and since the upgrade to Java17 nothing was rebooted.

[hmerk (Hans-Jörg Merk)]

OH runs out of memory

You can explicitly set the mx limit to a higher value or remove it then it’ll use the default which depends on RAM (I think for a 2GB ARM machine it’s 512m). Beware, however, that I haven’t yet seen a system that really needs that much heap. So if you keep hitting a 320m or even higher limit, it’s likely you have a mem leak. Increasing the limit further will then buy you some more time but it is not a solution in that case.

[Simon_B (Simon Ban)]

I got 6 GB on this VM running.
and set EXTRA_JAVA_OPTS=“-Xms2048m -Xmx4096m” and also set to ```
EXTRA_JAVA_OPTS=“-Xms128m -Xmx512m”
without any changes.

[berni]

You should not give Openhab much more memory than it actually needs. 4GB sounds way too much for me. You are probably fine with 1-2GB Xmx.
It’s the Linux kernel that killed the Java process, it’s not the Java JVM that said that it ran out of heap space.
Xmx is the max heap space, but that’s not the only type of memory Java needs, so the actual memory usage overall of Openhab can be (for example) 5GB if you set 4GB heap. And of course there are probably a lot of other processes on your Proxmox that also need memory and your Linux kernel killed the process that uses the most memory.
Maybe try out -Xmx2g, depending on your needs.

[berni]

Java is a memory hog, it will eat all the memory you give it until it comes close to the heap limit, then it starts seriously doing garbage collection and freeing up memory again which it doesn’t need anymore. But I think the JVM Garbage collector only watches its own Java process memory and doesn’t care if the whole Linux system is having memory pressure already.

[Simon_B (Simon Ban)]

i Have now downgraded to Java zulu 11.
I will test with: EXTRA_JAVA_OPTS=“-Xms128m -Xmx2g”

[Simon_B (Simon Ban)]

i got 16 GB on the Proxmox Server and only 2 VM with 6 GB und 4 GB, i will reduce RAM to 4 GB per System and check if the error shows up again.
Thanks for your reply!