Systemtcl restart fails

My girl friend is home alone this week and out of the blue openhab stopped running, she did cut the power to rpi several times to try to reboot it. In the end I managed to tell her how to obtain the IP of openhabian(openhabianpi stopped working) and she then managed to log on to the RPI and she sent me this:

**[15:40:28]**  **openhabian@openHABianPi** : **~** $ sudo systemctl restart openhab2.service

**[15:40:33]**  **openhabian@openHABianPi** : **~** $ sudo systemctl status openhab2.service

● openhab2.service - openHAB 2 - empowering the smart home

   Loaded: loaded (/usr/lib/systemd/system/openhab2.service; enabled; vendor pre

   Active: activating (auto-restart) (Result: exit-code) since Tue 2018-09-04 15



  Process: 23007 ExecStop=/usr/share/openhab2/runtime/bin/karaf stop  **(code=exite**

  Process: 22915 ExecStart=/usr/share/openhab2/runtime/bin/karaf $OPENHAB_STARTM

 Main PID: 22915 (code=exited, status=1/FAILURE)

Sep 04 15:41:55 openHABianPi systemd[1]:  **openhab2.service: Control process exite**

Sep 04 15:41:55 openHABianPi systemd[1]:  **openhab2.service: Unit entered failed s**

Sep 04 15:41:55 openHABianPi systemd[1]:  **openhab2.service: Failed with result 'e**

lines 1-12/12 (END)

What does these error message means, is there another way to restart the OH enviroment?
I ordered a second RPI now, and some ribbon cable so that I can have a switch to select which RPI to run in case one fails…

I’m going to guess that on one of the power pulls the file system became corrupted which messed up OH or Java. There isn’t enough here to really say what the problem is but if OH used to work, there were no changes, and then it stopped working then:

  • something actually changed that you are not aware of like a upgrade of some system packages
  • the file system became corrupt, usually caused by loss of power instead of a shutdown
  • the file system became corrupt because the SD card it failing.

I don’t think there is much you can do remotely to fix this unless you can remotely log in. You might be able to limp by by reinstalling OH with apt-get --force install openhab2 (double check that command, I’m going from memory) which might get it up and running again, unless the problem is Java and not OH itself.

Many people’s strategy to address this sort of problem is to have a spare SD card with a recent backup of OH. When the worst happens, train your family and guests to replace the SD card when necessary. That is cheaper and just about as easy as flipping a switch and running on two RPis.

My approach is to have the home automation fail gracefully. Even when my OH is offline, there is still the old manual way to control everything. I don’t run on an RPi so the SD card approach isn’t an option.

Good luck!

Yeah that would be the best, but my downlights are rgb, and connected to a dmx driver, and my heating is connected to relays. Any ideas how to avoid this? I was thinking about adding manual override switches for heating… But for the dmx i am a bit lost…

Everything except of OH is running nicely on the RPI, and it was a brand new SD card(like 1 month) and it just stopped out of the blue…

I have not used dmx, does it not have a manual controller? I am similar to Rich, I have manual on off even if all fails. I would not be able to dim or control with voice, but light still comes on.

Can you have the old mains switch maybe and set the dmx to come on after a power cycle? That way you had something in a failure.

On the heater relays similar question. How was the heater controlled before openHAB? Can that be used as a fail safe. I am trying to understand your setup to propose some ideas to help.

I do agree though automation is great, but there are a few issues. Mine is I have lights automated on in the morning. But not automated off as someone is always home. This next few days everyone is traveling overnight. Now what? Not looking for an answer here, merely an example.

Power outage prior to the failure? Most of the time when an SD card goes bad, it really hasn’t worn out but it became corrupted during the power loss. In this case the card is physically fine but some important files became corrupted because a write that was in progress or waiting to occur didn’t occur because of the power outage. The way SD cards work, multiple parts of multiple files live in the same block of storage. When any one of those files in that one block change, the entire block is rewritten over to a new block which includes those parts of files that didn’t change. When you lose power before the write finishes, those parts of all the files in the block can potentially be lost, incomplete, or otherwise corrupted. And those parts of files could be log files or parts of binaries that make up programs or even the Linux kernel.

I don’t have any advice for your heating or lighting as I don’t use either technology. Given my personal approach to home automation, I probably would not have chosen those technologies in the first place. Of course that doesn’t help you right now.

Unfortunately if you don’t already have remote connectivity to your RPi I don’t know if there is much you can do until you get home. Once you do get home, or if you feel like you can walk your gf through the steps you can try to force a reinstall of OH. If only OH files are corrupted, assuming this is the source of the problem, then those files will be replaced and you will be back up and running.

Then I suggest using the spare SD card approach so next time you can just swap out cards and be back up and running.

1 Like

True, have no Idea how I can remote access to it though, probably some port forwarding at the router is needed.

So in the meantime I have done some thinking, and as a start I have ordered a DMX controller panel so that I manual can control the lights, I might buy a wifi dongle for it so I can move it around in the flat later.

I also found out that there is some alternative(not OH) android app to control artnet dmx light, so I propably would install that as a backup as well. For lights connected to relay and heating cables I still need to do some thinking :slight_smile:

She didnt do that untill OH stopped working, so I doubt pulling the power triggered it, and everything works except of OH…

Final note I could also be responsible, because I tried to add a script to systemtcl a few weeks back, and I might have corrupted systemtcl, how should the opehab service file look like and where is it located? Is there manual way to start openhab?

On tuesday I will be back and try the

I will make a backup of the config catalogue before doing that, samba share is working fine, so I can easily drag the files over to my laptop!

SSH tunnels or OpenVPN would require port forwarding. But both are solid products and can be configured to require multi-factor authentication (i.e. type a password and present a certificate) so of all the things you would port forward these are least troublesome.

I actually have both.

There are other solutions though that rely on a cloud service to establish that initial connection, freeing you from needing to poke a hole in your firewall, but at the cost of reliance on a cloud service.

Years ago I used to use Hamachi. I use Teamviewer to remotely administer my dad’s computers now.

The RPi site lists a few other alternatives but I’ve no experience with those:

Description=openHAB 2 - empowering the smart home



ExecStart=/usr/share/openhab2/runtime/bin/karaf $OPENHAB_STARTMODE
ExecStop=/usr/share/openhab2/runtime/bin/karaf stop

SuccessExitStatus=0 143



Note that running OH this way requires you keep the terminal open. Once you close the terminal OH will be killed.

Make sure to grab the JSONDB files as well.

Just got home, seems like its java that its broken:

# A fatal error has been detected by the Java Runtime Environment:
#  SIGILL (0x4) at pc=0x7570dac8, pid=32406, tid=0x7526c470
# JRE version:  (8.0_152-b76) (build )
# Java VM: OpenJDK Client VM (25.152-b76 mixed mode, sharing, Evaluation linux-aarch32 )
# Problematic frame:
# V  []  os::Linux::libpthread_init()+0x0
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
# If you would like to submit a bug report, please visit:

---------------  T H R E A D  ---------------

Current thread is native thread

siginfo: si_signo: 4 (SIGILL), si_code: 1 (ILL_ILLOPC), si_addr: 0x7570dac8


Top of Stack: (sp=0x7526bbe8)
0x7526bbe8:   ffffffff 00000000 fffc0b26 7570c228
0x7526bbf8:   00000000 00000000 00f42f80 00000005
0x7526bc08:   00000000 758eee04 758cd158 7526bc50
0x7526bc18:   ffffffff 00ffffff 6d98a714 0029b19f
0x7526bc28:   00000005 00000000 7ef42000 7526bc90
0x7526bc38:   7526be38 00000000 00000000 00000001
0x7526bc48:   7526bc7c 7526bc58 7ef436cc 7ef4330c
0x7526bc58:   5b980886 2a82b489 7526bc90 00000000 

Instructions: (pc=0x7570dac8)
0x7570daa8:   62 8f 0c b8 76 2c 5f 93 e1 e1 9a 30 61 40 5a 5c
0x7570dab8:   6c 4e 56 03 ad ff 59 80 44 82 44 5d ac 5c ef 09
0x7570dac8:   11 ac 96 d7 6c ee 21 f1 33 91 de c5 b4 6c 8d 92
0x7570dad8:   e1 8b ba b4 72 13 74 c0 3f 28 f5 8b c0 b4 c0 b1 

Stack: [0x7521d000,0x7526d000],  sp=0x7526bbe8,  free space=314k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  []  os::Linux::libpthread_init()+0x0
V  []  Threads::create_vm(JavaVMInitArgs*, bool*)+0xe8
V  []  JNI_CreateJavaVM+0x8c
C  []  JavaMain+0x74
C  []  start_thread+0xb4

---------------  P R O C E S S  ---------------

VM state:not at safepoint (not fully initialized)

VM Mutex/Monitor currently owned by a thread: None

Dynamic libraries:
00008000-00019000 r-xp 00000000 b3:02 136321     /usr/lib/jvm/zulu-embedded-8-armhf/bin/java
00020000-00021000 rw-p 00010000 b3:02 136321     /usr/lib/jvm/zulu-embedded-8-armhf/bin/java
00021000-00023000 rw-p 00000000 00:00 0 
00f71000-00f92000 rw-p 00000000 00:00 0          [heap]
74ee0000-7509f000 rw-p 00000000 00:00 0 
7509f000-750d1000 r-xp 00000000 b3:02 136494     /usr/lib/jvm/zulu-embedded-8-armhf/jre/lib/aarch32/
750d1000-750d9000 ---p 00032000 b3:02 136494     /usr/lib/jvm/zulu-embedded-8-armhf/jre/lib/aarch32/
750d9000-750da000 rw-p 00032000 b3:02 136494     /usr/lib/jvm/zulu-embedded-8-armhf/jre/lib/aarch32/
750da000-750dc000 rw-p 00000000 00:00 0 
750dc000-750f6000 r-xp 00000000 b3:02 136470     /usr/lib/jvm/zulu-embedded-8-armhf/jre/lib/aarch32/
750f6000-750fd000 ---p 0001a000 b3:02 136470     /usr/lib/jvm/zulu-embedded-8-armhf/jre/lib/aarch32/
750fd000-750fe000 rw-p 00019000 b3:02 136470     /usr/lib/jvm/zulu-embedded-8-armhf/jre/lib/aarch32/
750fe000-75100000 rw-p 00000000 00:00 0 
75100000-75121000 rw-p 00000000 00:00 0 
75121000-75200000 ---p 00000000 00:00 0 
75206000-7520c000 r-xp 00000000 b3:02 1968       /lib/arm-linux-gnueabihf/
7520c000-7521b000 ---p 00006000 b3:02 1968       /lib/arm-linux-gnueabihf/
7521b000-7521c000 r--p 00005000 b3:02 1968       /lib/arm-linux-gnueabihf/
7521c000-7521d000 rw-p 00006000 b3:02 1968       /lib/arm-linux-gnueabihf/
7521d000-7521e000 ---p 00000000 00:00 0 
7521e000-7526d000 rw-p 00000000 00:00 0 
7526d000-752da000 r-xp 00000000 b3:02 1930       /lib/arm-linux-gnueabihf/
752da000-752ea000 ---p 0006d000 b3:02 1930       /lib/arm-linux-gnueabihf/
752ea000-752eb000 r--p 0006d000 b3:02 1930       /lib/arm-linux-gnueabihf/
752eb000-752ec000 rw-p 0006e000 b3:02 1930       /lib/arm-linux-gnueabihf/
752ec000-758b3000 r-xp 00000000 b3:02 136464     /usr/lib/jvm/zulu-embedded-8-armhf/jre/lib/aarch32/client/
758b3000-758ba000 ---p 005c7000 b3:02 136464     /usr/lib/jvm/zulu-embedded-8-armhf/jre/lib/aarch32/client/
758ba000-758e8000 rw-p 005c6000 b3:02 136464     /usr/lib/jvm/zulu-embedded-8-armhf/jre/lib/aarch32/client/
758e8000-76d0a000 rw-p 00000000 00:00 0 
76d0a000-76d26000 r-xp 00000000 b3:02 1918       /lib/arm-linux-gnueabihf/
76d26000-76d35000 ---p 0001c000 b3:02 1918       /lib/arm-linux-gnueabihf/
76d35000-76d36000 r--p 0001b000 b3:02 1918       /lib/arm-linux-gnueabihf/
76d36000-76d37000 rw-p 0001c000 b3:02 1918       /lib/arm-linux-gnueabihf/
76d37000-76e60000 r-xp 00000000 b3:02 1903       /lib/arm-linux-gnueabihf/
76e60000-76e70000 ---p 00129000 b3:02 1903       /lib/arm-linux-gnueabihf/
76e70000-76e72000 r--p 00129000 b3:02 1903       /lib/arm-linux-gnueabihf/
76e72000-76e73000 rw-p 0012b000 b3:02 1903       /lib/arm-linux-gnueabihf/
76e73000-76e76000 rw-p 00000000 00:00 0 
76e76000-76e78000 r-xp 00000000 b3:02 1912       /lib/arm-linux-gnueabihf/
76e78000-76e87000 ---p 00002000 b3:02 1912       /lib/arm-linux-gnueabihf/
76e87000-76e88000 r--p 00001000 b3:02 1912       /lib/arm-linux-gnueabihf/
76e88000-76e89000 rw-p 00002000 b3:02 1912       /lib/arm-linux-gnueabihf/
76e89000-76ead000 r-xp 00000000 b3:02 135718     /usr/lib/jvm/zulu-embedded-8-armhf/lib/aarch32/jli/
76ead000-76eb4000 ---p 00024000 b3:02 135718     /usr/lib/jvm/zulu-embedded-8-armhf/lib/aarch32/jli/
76eb4000-76eb5000 rw-p 00023000 b3:02 135718     /usr/lib/jvm/zulu-embedded-8-armhf/lib/aarch32/jli/
76eb5000-76eb7000 rw-p 00000000 00:00 0 
76eb7000-76ecd000 r-xp 00000000 b3:02 1964       /lib/arm-linux-gnueabihf/
76ecd000-76edc000 ---p 00016000 b3:02 1964       /lib/arm-linux-gnueabihf/
76edc000-76edd000 r--p 00015000 b3:02 1964       /lib/arm-linux-gnueabihf/
76edd000-76ede000 rw-p 00016000 b3:02 1964       /lib/arm-linux-gnueabihf/
76ede000-76ee0000 rw-p 00000000 00:00 0 
76ee0000-76ee5000 r-xp 00000000 b3:02 10591      /usr/lib/arm-linux-gnueabihf/
76ee5000-76ef4000 ---p 00005000 b3:02 10591      /usr/lib/arm-linux-gnueabihf/
76ef4000-76ef5000 r--p 00004000 b3:02 10591      /usr/lib/arm-linux-gnueabihf/
76ef5000-76ef6000 rw-p 00005000 b3:02 10591      /usr/lib/arm-linux-gnueabihf/
76ef6000-76f17000 r-xp 00000000 b3:02 1855       /lib/arm-linux-gnueabihf/
76f18000-76f1a000 rw-p 00000000 00:00 0 
76f22000-76f23000 r--p 00000000 00:00 0 
76f23000-76f26000 rw-p 00000000 00:00 0 
76f26000-76f27000 r--p 00020000 b3:02 1855       /lib/arm-linux-gnueabihf/
76f27000-76f28000 rw-p 00021000 b3:02 1855       /lib/arm-linux-gnueabihf/
7eb9a000-7ebbb000 rw-p 00000000 00:00 0          [stack]
7ef41000-7ef42000 r-xp 00000000 00:00 0          [sigpage]
7ef42000-7ef43000 r--p 00000000 00:00 0          [vvar]
7ef43000-7ef44000 r-xp 00000000 00:00 0          [vdso]
ffff0000-ffff1000 r-xp 00000000 00:00 0          [vectors]

VM Arguments:
java_command: <unknown>
java_class_path (initial): .
Launcher Type: SUN_STANDARD

Environment Variables:

Signal Handlers:
SIGSEGV: [], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGBUS: [], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGFPE: [], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGPIPE: [], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGXFSZ: [], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGILL: [], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGUSR1: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
SIGUSR2: [], sa_mask[0]=00000000000000000000000000000000, sa_flags=SA_RESTART|SA_SIGINFO
SIGHUP: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
SIGINT: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
SIGTERM: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
SIGQUIT: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none

---------------  S Y S T E M  ---------------

OS:PRETTY_NAME="Raspbian GNU/Linux 9 (stretch)"
NAME="Raspbian GNU/Linux"
VERSION="9 (stretch)"

uname:Linux 4.14.52-v7+ #1123 SMP Wed Jun 27 17:35:49 BST 2018 armv7l
libc:(null) (null) (fixed stack)
rlimit: STACK 8192k, CORE 0k, NPROC 7741, NOFILE 1024, AS infinity
load average:0.24 0.06 0.05

MemTotal:        1000184 kB
MemFree:           49788 kB
MemAvailable:     820808 kB
Buffers:           69276 kB
Cached:           728020 kB
SwapCached:            0 kB
Active:           315808 kB
Inactive:         520508 kB
Active(anon):      41816 kB
Inactive(anon):    47876 kB
Active(file):     273992 kB
Inactive(file):   472632 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:        102396 kB
SwapFree:         102396 kB
Dirty:                56 kB
Writeback:             0 kB
AnonPages:         39060 kB
Mapped:            70936 kB
Shmem:             50676 kB
Slab:             101772 kB
SReclaimable:      89932 kB
SUnreclaim:        11840 kB
KernelStack:        1088 kB
PageTables:         1988 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:      602488 kB
Committed_AS:     355176 kB
VmallocTotal:    1064960 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
CmaTotal:           8192 kB
CmaFree:            6792 kB

CPU:total 4 (initial active 4) 

processor	: 0
model name	: ARMv7 Processor rev 4 (v7l)
BogoMIPS	: 76.80
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd03
CPU revision	: 4

processor	: 1
model name	: ARMv7 Processor rev 4 (v7l)
BogoMIPS	: 76.80
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd03
CPU revision	: 4

processor	: 2
model name	: ARMv7 Processor rev 4 (v7l)
BogoMIPS	: 76.80
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd03
CPU revision	: 4

processor	: 3
model name	: ARMv7 Processor rev 4 (v7l)
BogoMIPS	: 76.80
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd03
CPU revision	: 4

Hardware	: BCM2835
Revision	: a02082
Serial		: 00000000c6cdb751

Memory: 4k page, physical 1000184k(49788k free), swap 102396k(102396k free)

vm_info: OpenJDK Client VM (25.152-b76) for linux-aarch32 JRE (1.8.0_152-b76), built on Oct 20 2017 14:26:08 by "jenkins" with gcc 4.8.3 20140303 (prerelease)

time: Tue Sep 11 20:25:10 2018
elapsed time: 0 seconds (0d 0h 0m 0s)

Indeed it does. We can’t say for sure if it was the original cause of the outage or if it became corrupted later, but that certainly looks like a core dump from the JVM.

I am copying out the share directory now, takes forever it 35000 files… Is it possible to upgrade or reinstall just the java?

Now i reinstalled java, however I get the following error:

[01:14:21] openhabian@openHABianPi:~$ sudo /usr/share/openhab2/
Launching the openHAB runtime...
mkdir: cannot create directory ‘/usr/share/openhab2/userdata/tmp’: No such file or directory
KARAF_BASE is not valid: /usr/share/openhab2/userdata

what does this error mean?

My directory looks like this

Use sudo openhab-cli start to start a temporary instance of openHAB if you’re using openHABian or the apt packages. Calling directly expects the files to remain in one location.

That said, if JAVA was responsible for the outage, sudo systemctl start openhab2 should suffice now that it is fixed.

1 Like

Thanks @Benjy,
I think the problem was not java in the end.

told me openhab was running under root and not as openhab user, and the error seemed like it were java, but I think it was just the fact that it was running under root.

This happened because I tried to restart openhab with an item:

Switch Div_Openhab {exec="ON:sudo systemctl restart opeanhab2.service"}

Maybe some of the maintainer can make a change to service so that it can not be started as root user…

sudo openhab-cli start will tell you which prosess to kill : sudo kill pid 520 if someone else experience the same issue.

So I still can not get Openhab to lunch with systemctl, so I guess its corrupted or something. I can not use sudo -openhab-cli start because then it shut down as soon as I close my laptop. So can I reinstall systemctl? Because the service file looks just like Rich pointed out:
sudo apt-get reinstall systemd

What do the commands:

sudo systemctl status openhab2
sudo journalctl -u openhab2.service -b
sudo openhab-cli info

tell you?

Did the openhab user have permission to run as a sudoer beforehand?