PS: A very important point I forgot to mention: the “Sampler” tab of the VisualVM tools shows the amount of memory allocations, not the used memory (which would be the allocations minus “deallocations”). So it’s not really sure that this high number really points to the leak!
Thanks for the detailed explanation. I also have access to another Openhab 2 instance in a separate house also running on a RPI that also leaks memory. I have no JMX connection there yet, but this installation uses much less addons as mine. These are the addons that both systems run:
Exec
Systeminfo
Z-Wave
I could do without the systeminfo addon and maybe without the exec addon, but without the Z-Wave addon my house doesn’t work any more.
You wrote that the memory allocations are not the used memory. But I didn’t find a column with memory deallocations. How can I find out the real memory usage of every thread?
I also found a thread that describes how a thread in a java process could be killed:
https://web.archive.org/web/20160302023213/http://www.rhcedan.com/2010/06/22/killing-a-java-thread
Maybe by killing the thread I would see if after that memory is released. (I hope the thread pool will just restart the thread after that) Or don’t you think that would work?
By the way, here are the informations you requested earlier:
# process statistics
cat /proc/1161/status
Name: java
Umask: 0022
State: S (sleeping)
Tgid: 1161
Ngid: 0
Pid: 1161
PPid: 626
TracerPid: 0
Uid: 109 109 109 109
Gid: 114 114 114 114
FDSize: 512
Groups: 5 20 44 50 100 114 997 998
NStgid: 1161
NSpid: 1161
NSpgid: 626
NSsid: 626
VmPeak: 704024 kB
VmSize: 701860 kB
VmLck: 0 kB
VmPin: 0 kB
VmHWM: 470228 kB
VmRSS: 467176 kB
RssAnon: 462636 kB
RssFile: 4540 kB
RssShmem: 0 kB
VmData: 628672 kB
VmStk: 132 kB
VmExe: 4 kB
VmLib: 9456 kB
VmPTE: 722 kB
VmPMD: 0 kB
VmSwap: 55456 kB
Threads: 233
SigQ: 0/7695
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000010000000
SigCgt: 2000000181005ccf
CapInh: 0000000000000000
CapPrm: 0000000000003400
CapEff: 0000000000003400
CapBnd: 0000003fffffffff
CapAmb: 0000000000000000
Seccomp: 0
Cpus_allowed: f
Cpus_allowed_list: 0-3
Mems_allowed: 1
Mems_allowed_list: 0
voluntary_ctxt_switches: 4
nonvoluntary_ctxt_switches: 9
# physical memory situation
free -h
total used free shared buffers cached
Mem: 970M 882M 88M 3.0M 39M 180M
-/+ buffers/cache: 662M 308M
Swap: 99M 99M 48K
# garbage collector statistics
jstat -gccapacity 1161
NGCMN NGCMX NGC S0C S1C EC OGCMN OGCMX OGC OC MCMN MCMX MC CCSMN CCSMX CCSC YGC FGC
5440.0 83264.0 76992.0 7680.0 7680.0 61632.0 10944.0 166592.0 152840.0 152840.0 0.0 70960.0 70704.0 0.0 0.0 0.0 59969 14
jstat -gcutil 1161
S0 S1 E O M CCS YGC YGCT FGC FGCT GCT
0.00 4.91 94.11 69.20 81.25 - 59969 1291.529 14 14.605 1306.134
# JVM arguments
sudo -H -u openhab bash -c 'jcmd 1161 VM.command_line'
1161:
VM Arguments:
jvm_args: -Dopenhab.home=/usr/share/openhab2 -Dopenhab.conf=/etc/openhab2 -Dopenhab.runtime=/usr/share/openhab2/runtime -Dopenhab.userdata=/var/lib/openhab2 -Dopenhab.logdir=/var/log/openhab2 -Dfelix.cm.dir=/var/lib/openhab2/config -Djetty.host=0.0.0.0 -Dorg.ops4j.pax.web.listening.addresses=0.0.0.0 -Dorg.osgi.service.http.port=8080 -Dorg.osgi.service.http.port.secure=8443 -Djava.awt.headless=true -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9010 -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=172.17.2.21 -Dcom.sun.management.jmxremote.rmi.port=9010 -Djava.endorsed.dirs=/usr/lib/jvm/zulu-embedded-8-armhf/jre/lib/endorsed:/usr/lib/jvm/zulu-embedded-8-armhf/lib/endorsed:/usr/share/openhab2/runtime/lib/endorsed -Djava.ext.dirs=/usr/lib/jvm/zulu-embedded-8-armhf/jre/lib/ext:/usr/lib/jvm/zulu-embedded-8-armhf/lib/ext:/usr/share/openhab2/runtime/lib/ext -Dkaraf.instances=/usr/share/openhab2/runtime/instances -Dkaraf.home=/usr/share/openhab2/runtime -Dkaraf.base=/var/lib/openhab2 -Dkaraf.data=/var/lib/openhab2 -Dkaraf.etc=/var/lib/openhab2/etc -Dkaraf.restart.jvm.supported=true -Djava.io.tmpdir=/var/lib/openhab2/tmp -Djava.util.logging.config.file=/var/lib/openhab2/etc/java.util.logging.properties -Dkaraf.startLocalConsole=false -Dkaraf.startRemoteShell=true
java_command: org.apache.karaf.main.Main
java_class_path (initial): /usr/share/openhab2/runtime/lib/boot/org.apache.karaf.diagnostic.boot-4.0.8.jar:/usr/share/openhab2/runtime/lib/boot/org.apache.karaf.jaas.boot-4.0.8.jar:/usr/share/openhab2/runtime/lib/boot/org.apache.karaf.main-4.0.8.jar:/usr/share/openhab2/runtime/lib/boot/org.osgi.core-6.0.0.jar
Launcher Type: SUN_STANDARD
sudo -H -u openhab bash -c 'jcmd 1161 VM.flags'
1161:
-XX:InitialHeapSize=16777216 -XX:+ManagementServer -XX:MaxHeapSize=255852544 -XX:MaxNewSize=85262336 -XX:MinHeapDeltaBytes=131072 -XX:NewSize=5570560 -XX:OldSize=11206656