openHAB 2 and 3 crash with SIGSEGV

Setup, Configuration and Use Runtime
1

  • Platform information A:

    • Hardware: VirtualBox headless VM with 1 CPU, 1024 MB on host i5-8400 16 GB Debian 11
    • OS: Debian 10
    • Java Runtime Environment: zulu8-jre OpenJDK Runtime Environment (Zulu 8.66.0.15-CA-linux64) (build 1.8.0_352-b08)
    • openHAB version: 2.5.12 Release build

  • Platform information B:

    • Hardware: VirtualBox headless VM with 2 CPU, 2048 MB on same i5-8400 host
    • OS: Debian 11
    • Java Runtime Environment: openjdk-11-jre OpenJDK Runtime Environment (build 11.0.16+8-post-Debian-1deb11u1)
    • openHAB version: 3.3.0 Release Build

  • Issue of the topic:

    • I have two instances of openHAB running in VMs on the same host, my production system with openHAB 2.5, and a new openHAB 3.3 installation intended to replace the old one, eventually,
    • both instances crash approx. 1-2 times per day, with a SIGSEGV. An example log is shown below.
    • there are other VMs running on the same host, also under Debian 10 or 11, they never crash
    • occasionally other programs on the VMs running openHAB have also crashed with SIGSEGV, e.g. influxdb or a custom-made python script. I don’t see such crashes on other non-openHAB VM on the same host

  • Configurations (if applicable):
    Can’t post all configuration files, with 120 things, 956 items, 77 rules, but the crash doesn’t appear to be linked to a specific item. I checked whether e.g. the same item was always updated, or the same rule triggered, immediately before the crash, but that is not the case

  • Logs:
    A typical log, from the openHAB 3.3 machine

2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -  #
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -  # A fatal error has been detected by the Java Runtime Environment:
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -  #
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -  #  SIGSEGV (0xb) at pc=0x00007fdaa0ef182e, pid=481443, tid=488256
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -  #
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -  # JRE version: OpenJDK Runtime Environment (11.0.16+8) (build 11.0.16+8-post-Debian-1deb11u1)
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -  # Java VM: OpenJDK 64-Bit Server VM (11.0.16+8-post-Debian-1deb11u1, mixed mode, tiered, compressed oops, g1 gc, linux-amd64)
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -  # Problematic frame:
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -  # J 23611 c2 java.util.concurrent.LinkedTransferQueue.xfer(Ljava/lang/Object;ZIJ)Ljava/lang/Object; java.base@11.0.16 (236 bytes) @ 0x00007fdaa0ef182e [0x00007fdaa0ef14e0+0x000000000000034e]
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -  #
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -  # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -  #
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -  # An error report file with more information is saved as:
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -  # /var/lib/openhab/hs_err_pid481443.log
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -  Compiled method (nm) 10367055 2638     n 0       java.lang.Thread::isAlive (native)
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -   total in heap  [0x00007fdaa03cb690,0x00007fdaa03cba20] = 912
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -   relocation     [0x00007fdaa03cb808,0x00007fdaa03cb838] = 48
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -   main code      [0x00007fdaa03cb840,0x00007fdaa03cba20] = 480
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -  Compiled method (c2) 10367056 17516   !   4       java.util.concurrent.ThreadPoolExecutor::getActiveCount (70 bytes)
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -   total in heap  [0x00007fdaa0a52110,0x00007fdaa0a53bc0] = 6832
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -   relocation     [0x00007fdaa0a52288,0x00007fdaa0a52378] = 240
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -   main code      [0x00007fdaa0a52380,0x00007fdaa0a53320] = 4000
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -   stub code      [0x00007fdaa0a53320,0x00007fdaa0a53380] = 96
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -   metadata       [0x00007fdaa0a53380,0x00007fdaa0a53458] = 216
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -   scopes data    [0x00007fdaa0a53458,0x00007fdaa0a53808] = 944
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -   scopes pcs     [0x00007fdaa0a53808,0x00007fdaa0a53a68] = 608
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -   dependencies   [0x00007fdaa0a53a68,0x00007fdaa0a53a78] = 16
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -   handler table  [0x00007fdaa0a53a78,0x00007fdaa0a53b68] = 240
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -   nul chk table  [0x00007fdaa0a53b68,0x00007fdaa0a53bc0] = 88
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -  Could not load hsdis-amd64.so; library not loadable; PrintAssembly is disabled
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -  #
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -  # If you would like to submit a bug report, please visit:
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -  #   https://bugs.debian.org/openjdk-11
2022-11-29T04:16:21+01:00 ha-server3 karaf 481443 - -  #
2022-11-29T04:16:21+01:00 ha-server3 systemd 1 - -  openhab.service: Main process exited, code=killed, status=6/ABRT
2022-11-29T04:16:21+01:00 ha-server3 systemd 1 - -  openhab.service: Failed with result 'signal'.
2022-11-29T04:16:21+01:00 ha-server3 systemd 1 - -  openhab.service: Consumed 4min 26.202s CPU time.
2022-11-29T04:16:26+01:00 ha-server3 systemd 1 - -  openhab.service: Scheduled restart job, restart counter is at 2.
2

It’s not openHAB crashing, it’s your Java crashing. “Your” because you selected and installed it.
“Your” also because it runs on some Linux machine that noone but yourself knows how it is setup.
The log shows Could not load hsdis-amd64.so as the reason your JVM instance terminates itself so seems there some files missing.
This does not happen on a fresh install of OpenJDK 11 on a genuine Debian installation so I don’t think there’s advice we can give other than to reinstall Java, and if that doesn’t fix it, reinstall your OS.

3

Does the error report stored in

provide more details ?

4

I attach the file here, this is from a crash of the openHAB 3 instance.
hs_err_pid481443.log (146.9 KB)
I also attach a log file from a recent crash of the openHAB 2 instance
hs_err_pid7410.log (92.5 KB)

I don’t know how to interpret these, maybe someone can help?

5

I have seen cases where forum members have been able to help with problems, despite the fact that the problem happened on my computer, and noone but myself knows how it is setup … :slight_smile:

Seriously … of course “Java crashed”, because openHAB is written in Java. The message about “hsdis-amd64.so” doesn’t always occur. The fact remains that I get these crashes with two different openHAB versions, running on two different Java runtime environments

How can I tell that it is a “genuine” Debian installation and not a fake?

Regarding the advice to “reinstall Java, and reinstall the OS”, one might argue that I have already done that: create a new VM, install latest Debian (hopefully “genuine”), and latest openHAB, and the JRE recommended by openHAB. After seeing crashes, I un-installed zulu JRE and installed a new JRE, this time openjdk-11-jre, only to see more crashes

6

I’d be making some basic hardware integrity checks; power supply voltages and noise, memory test.

7

Which doesn’t mean anything except that you have at least two different symptoms and/or causes you need to distinguish when looking after them.
The fact remains that it is never the Java app (openHAB) to cause JVM crashes but your JVM installation(s). That wording is explicitly including all of your OS setup and environment parameters you run OH/Java with.
It doesn’t happen elsewhere so it’s specific to and ultimately caused by what you installed and how you did or didn’t do during setup.

Then you should start by comprehensively sorting your various issues and just go for one of them first.
Don’t expect any magic help but provide more detailed information if you want others to help you solve your problem. The message points at a missing file so search for that one on your box and go google for the term. Try a different Java version package version, see if that contains that file.
None of this is really related to openHAB.
All of this is generic IT best practice and expected from any user him/herself.

8

I would try to update your host OS and VirtualBox so they use the latest fixes. I’ve seen strange JVM crashes before due to virtualization issues. :confused:

9

Thanks for sharing your experience with virtualization, this has been helpful.

I have upgraded VirtualBox from 6.1.26 to 6.1.40 three days ago, and haven’t had any crashed VMs since.

1 Like
10

Is the host box a workstation that you also use for other desktop tasks or is it used as a server? The reason I ask is that I use Proxmox which is based on Debian and it’s been rock solid for me.

I appreciate that this doesn’t really help you with your immediate problem though :slight_smile:

11

I always try to prevent running “heavy” VMs (because Docker exists :whale:), but when I do, I prefer KVM, because its part of many distro’s. It’s very flexible and you can also run KVM in Docker containers which is sometimes useful in CI builds. :slight_smile:

Those JVM virtualization issues I ran into occured with VMware Player once upon a time. But all software is prone to bugs ofcourse. :upside_down_face: