How can I find a memory leak?

I’m using OH3.4.0.RC1 in a docker. I’m loosing about 200MB RAM every day. Where to search? The bindings I’m using are not special (Astro, Hue, Modbus, NTP, OpenWeatherMap, Remote OpenHAB, Shelly, Sonos and Telegram). Is a binding the issue our can it be in config file? Can I log the memory used by a binding? Any suggestions?

I don’t think so

Are those all the bindings you are using?
Do you use VScode to edit files?
how big is system? number of things items rules Main UI or file based
what language are rules written in and UI rules or file based

Does the RAM go back down or continue to get larger until crash?
Only way is to unload bindings one by one or unload them all and begin adding one by one as you monitor

Is there a reason you can’t upgrade from 3.4.0.RC1 to 3.4.0? The memory leak might be fixed, that’s quite an old version now.

First of all, why do you think you have a memory leak? what is the error, issue or symptom you are seeing? It is important to diagnose first before looking for answers, and any errors you post will also help others with the same issue find your post and solution.

What tells you this, how are you measuring? Linux does not free up ram unless it is needed, as it may be needed again, so unless the ram is actually full, it will not free up the ram. The free -h command on the linux terminal needs to be read correctly.

No not on a per binding basis. Your best to log with the system info binding the channel Systeminfo_UsedHeapPercent. The entire JAVA environment uses a heap of ram called the heap and does not use ram like a normal program. Once again, the heap needs to be interrupted correctly as it is not freed up unless it is actually needed. Can you show what the heap history looks like after at least 24 hours of graphing its values?

Start unusing bindings, i.e. remove bindings from config, only one each time, and check if memory leakage comes to a halt. Shelly, Sonos would be prime candidates.

On the other hand, these are well established modules in widespread use.
So like others already did, I’d challenge your statement you’re loosing RAM first. Docker is just complicating the setup and is an additional potential source for problems like this.

I am using OH 3.4.0 in Docker and I also can see the memory occupied by OH is increasing every day. When I restart OH the memory consumption is back to normal. I did not see that pattern in OH 3.3.0 also running in Docker with the exact same bindings.

In the log I can see these messages, and I do not know if it is related:

2022-12-31 07:39:08.973 [WARN ] [ache.cxf.phase.PhaseInterceptorChain] - Interceptor for {http://internal.id.core.openhab.org/}UUIDResource has thrown exception, unwinding now

org.apache.cxf.interceptor.Fault: Could not send Message

After that many, many derived error lines are shown in the log - some are referring to Java.

I guess I will also try to remove the bindings one by one, when I have better time. For now I just restart at regular intervals.

Hi,
So, everything that has been mentioned above are all good steps to isolating the possible cause as well as creating a good problem statement for your issue.
As for actually finding the leak that is a completely different effort. For that you will need to use a debugger tool.
This article outlines what built in tools are available within the java JDK that your Openhab is running in.

That plus some googling and you will see what has to be done to actually track the memory leak (greedy or lazy memory consumer) down to a specific allocation or thread that is not releasing memory when it is finished using it.
Good luck.

The error message in the log is probably related to some unknown error in iOS app. I will stop using that app to identify if this is the reason for increased memory consumption.
I am using these bindings: hue, mqtt, modbus, rfxcom, ihc, exec, velux, pushover and deconz + persistence in Postgres using jdbc. Finally, I am using openhabcloud and homekit. This means both me and @HenMus use hue and modbus binding. @HenMus : are you using Postgres for persistence, openhabcloud or homekit?

I can also observe that OH 3.4.0 seems to be more rigid with number conversation types than OH 3.3.0. Now I need to specify the exact type format more places in several DSL rule files. I am doing this change, as the errors pop up into the log file. However, I am not aware if this could lead to an increase in memory over time.

I have now added additional exact number type formattering plus some extra tests for UNDEF in my DSL Rule files. Also, I have stopped using the iOS openhab app. As a result I got a OH log file without any errors.

After that I disabled binding modbus, but OH still increases memory consumption every day. I cannot easily disable other bindings or features (perist to postgres, openhabcloud and homekit), because then our house with 5 people onboard will no longer work properly.

It seems that the Java version in the Docker image is Temurin, but I do not know if that makes any difference. See below. I thought the default Java version in openhab was Azul Zulu.

image

If anyone could share some light on this, it would be great.

@justaoldman : you provided some links to Oracle Java for troubleshooting, but I guess that cannot be used for Temurin Java version. And even if it did, I would have no clue what do to and look for.

Except for these additional explicit number formatting and tests for UNDEF all DSL Rule files and bindings are identical to my OH 3.3.0 which ran without problems in Docker.

Any input would be appreciated. Thanks.

Ok so here and here are the links for tools for termurin versions .
First link is RH gives bit of insight and also links to other tools available.
Second is for the mission control and other tools you can use outside of the Eclipse IDE to debug.

As for how to debug a memory issue lots of google is your friend searching is best offer I can give you .
As for not wanting to impact the family and the household for that have you considered standing up a test environment and using it to figure out why this is happening.
Or perhaps stand up an alternate install of Openhab (perhaps the Linux flavor or heck even a windows version that you allow to manage the automation why you get this docker challenge puzzled out.
Maybe run the house on it while you are debugging this docker approach

Regards, Justan

Actually, I have been utilizing Docker to run several OPENHAB instances side by side. However, I have stopped it to rule out this as a root cause for my memory problems. The reason is that the OH web interface seems not to be designed for several instances even though different port numbers are used. Something with login token.

And it is not so easy to just spin up another instance to run the house while debugging. After all, they point to the same hardware devices and not all can handle e.g. 2 clients logging in at the same time.

Others seem to have similar problem in Docker - see here

I found a solution by adding --memory=1600m to my Docker run statement, which limits the memory to 1.6 GB. With this enabled OH3 starts around 0.9 GB in my setup gradually increasing to 1.1GB. After this it is stable, and OH3 does not request more memory.