[SOLVED] /var/lib/openhab2 gone

I’m inclined to agree with @kriznik here, a command shutdown or reboot (which will stop these services) should absolutely not cause loss or reset of data.

If the command you use truely properly call service shutdown scripts, yes. Reboot usually doesn’t.

ok @mstormi please share with us that secret command which will not corupt openhabian zram

because I tend to believe relevant way to reboot linux is shutdown -r aka reboot

i’ve stopped zram manually before reboot as well resulting in same corrupted OH as without

It’s the same binary yes but behavior depends on how it’s called. It’s confusing I know. But just type in those two commands and you should notice the difference (shutdown -r will wait for a minute).

how exactly? did you check if files were there and zram really was gone ?

I’m certainly sure that this is not the issue
but for another science I’ve done what you recommended:

openhabian@openhab:~ $ openhab-cli info

Version:     2.5.0.M2 (Build)

User:        openhab (Active Process 3288)
User Groups: openhab tty dialout audio bluetooth gpio

Directories: Folder Name      | Path                        | User:Group
             -----------      | ----                        | ----------
             OPENHAB_HOME     | /usr/share/openhab2         | openhab:openhab
             OPENHAB_RUNTIME  | /usr/share/openhab2/runtime | openhab:openhab
             OPENHAB_USERDATA | /var/lib/openhab2           | openhab:openhab
             OPENHAB_CONF     | /etc/openhab2               | openhab:openhab
             OPENHAB_LOGDIR   | /var/log/openhab2           | openhab:openhab

URLs:        http://10.10.1.5:8080
             https://10.10.1.5:8443

openhabian@openhab:~ $ sudo shutdown -r
Shutdown scheduled for Sat 2019-08-10 10:42:43 CEST, use 'shutdown -c' to cancel.
openhabian@openhab:~ $ Connection to 10.10.1.5 closed by remote host.
Connection to 10.10.1.5 closed.
kriznik@GMNG:~$ tv
Linux openhab 4.19.58-v7+ #1245 SMP Fri Jul 12 17:25:51 BST 2019 armv7l

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Fri Aug  9 20:39:40 2019 from 10.10.1.100
openhabian@openhab:~ $ openhab-cli info

Version:     2.4.0 (Build)

User:        openhab (Active Process 588)
User Groups: openhab tty dialout audio bluetooth gpio

Directories: Folder Name      | Path                        | User:Group
             -----------      | ----                        | ----------
             OPENHAB_HOME     | /usr/share/openhab2         | openhab:openhab
             OPENHAB_RUNTIME  | /usr/share/openhab2/runtime | openhab:openhab
             OPENHAB_USERDATA | /var/lib/openhab2           | root:root
             OPENHAB_CONF     | /etc/openhab2               | openhab:openhab
             OPENHAB_LOGDIR   | /var/log/openhab2           | openhab:openhab

URLs:        http://10.10.1.5:8080
             https://10.10.1.5:8443

openhabian@openhab:~ $

and 2.4 again

Yes, I‘ve enabled ZRAM (why? :slight_smile:). Rebooting was done with reboot command from ssh. Also toggling the power(I know that this is bad) caused same effect.
BTW might be worth adding correct reboot command to dashboard, like many routers are doing? It was first thing I looked in configuration page and found nothing.

Good because that’s what I’m worried about. Just tested on my own box with a fresh install from image and it works as designed (BTW even with “reboot” but that does not mean you should use it).

See above. You mustn’t use “reboot” or even pull the power plug.

Definitely not. openHABian is not a router or anything similar. It’s producing data that needs to be safely and permanently stored which is why you must shutdown it correctly to sync that data.
And you shouldn’t reboot at all. Under normal circumstances there is no need to, openHAB(ian) is designed to run 24/7.
Adding that to the dashboard would just make more people hit the button without a need to (which usually creates more trouble than it resolves).

well this is as well fresh instalation from yesterday, only thing which I’ve done to it is changed repo to 2.5M2 & zram enable & upgrade

So… that being said, what is troubleshoot procedure now?

Even if OH doesn‘t need reboots or shutdowns, the other services and hardware, connected to same Server might require it, especially when one changes settings and configs. In my case after it I have to take Z-stick and SD card out for backup, so I will need to shutdown OH properly.

So what is the right command?

You’re funny. This is not a commercial service with proven procedures let alone on an experimental feature…

I’m aware of that, sometimes it’s enough to say “I don’t know” but as you tried to point out that I’m not capable of rebooting properly I was keen to see what your magic solution is.

well, I’ll figure it out self then

(and because it is happening on two independed rpis running zram and 1.5openhabian I hardly believe your’s magically works out of the box)

Have you tried a different sd card?

I have several, and usb’s

it does not matter if it is sd card or usb boot (rpi3 and rpi3+) every instalation act exactly same after reboot

edit, stopping z-ram service won’t sync data either, so it is not really about reboot itself

openhabian@openhab:~ $ sudo openhab-cli info

Version:     2.5.0.M2 (Build)

User:        openhab (Active Process 2109)
User Groups: openhab tty dialout audio bluetooth gpio

Directories: Folder Name      | Path                        | User:Group
             -----------      | ----                        | ----------
             OPENHAB_HOME     | /usr/share/openhab2         | openhab:openhab
             OPENHAB_RUNTIME  | /usr/share/openhab2/runtime | openhab:openhab
             OPENHAB_USERDATA | /var/lib/openhab2           | openhab:openhab
             OPENHAB_CONF     | /etc/openhab2               | openhab:openhab
             OPENHAB_LOGDIR   | /var/log/openhab2           | openhab:openhab

URLs:        http://10.10.1.158:8080
             https://10.10.1.158:8443

openhabian@openhab:~ $ df
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        14G  2.3G   12G  17% /
devtmpfs        484M     0  484M   0% /dev
tmpfs           488M     0  488M   0% /dev/shm
tmpfs           488M  7.6M  481M   2% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           488M     0  488M   0% /sys/fs/cgroup
overlay1        372M   60M  284M  18% /var/lib/openhab2
/dev/sda1       253M   40M  213M  16% /boot
/dev/zram1      372M   60M  284M  18% /opt/zram/zram1
/dev/zram2      275M  1.5M  253M   1% /opt/zram/zram2
overlay2        275M  1.5M  253M   1% /var/log
tmpfs            98M     0   98M   0% /run/user/1000
openhabian@openhab:~ $ sudo service zram-config stop
openhabian@openhab:~ $ df
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        14G  2.3G   12G  17% /
devtmpfs        484M     0  484M   0% /dev
tmpfs           488M     0  488M   0% /dev/shm
tmpfs           488M  7.6M  481M   2% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           488M     0  488M   0% /sys/fs/cgroup
/dev/sda1       253M   40M  213M  16% /boot
/dev/zram1      372M   60M  284M  18% /opt/zram/zram1
/dev/zram2      275M  1.5M  253M   1% /opt/zram/zram2
tmpfs            98M     0   98M   0% /run/user/1000
openhabian@openhab:~ $ sudo openhab-cli info

Version:     2.4.0 (Build)

User:        openhab (Active Process 2109)
User Groups: openhab tty dialout audio bluetooth gpio

Directories: Folder Name      | Path                        | User:Group
             -----------      | ----                        | ----------
             OPENHAB_HOME     | /usr/share/openhab2         | openhab:openhab
             OPENHAB_RUNTIME  | /usr/share/openhab2/runtime | openhab:openhab
             OPENHAB_USERDATA | /var/lib/openhab2           | openhab:openhab
             OPENHAB_CONF     | /etc/openhab2               | openhab:openhab
             OPENHAB_LOGDIR   | /var/log/openhab2           | openhab:openhab

URLs:        http://10.10.1.158:8080
             https://10.10.1.158:8443

I tried shutdown command and also Markus‘s suggested (stopping service, unmount). Nothing changes - after rebooting /var/lib/openhab2 has root privileges and all is gone.

We’re talking openHABian here, and it does not require reboot. If you deliberately choose to deviate and do your own stuff then you do so at your own risk, you mustn’t expect openHABian to take care of that as well.
And it has a backup tool included so no need to take out your SD card, and any Z-stick can be safely removed so again, no need for reboot.

it’s indeed nonsense, there is a need of reboot time to time, and service simply can’t be broken because of that.

But in this case, it’s not issue at userpoint I believe as we can see in logs that:

rw,noatime
mount: /dev/zram2 mounted on /opt/zram/zram2.
mount: overlay2 mounted on /var/log.
createZlog no oldlog dir in ztab
zram-config stop 2019-08-10-11:53:24
ztab remove log /zram2 /var/log /log.bind
/zram2
Warning: Stopping rsyslog.service, but it can still be activated by:
  syslog.socket
umount: /var/log (overlay2) unmounted
overlay --lowerdir=/opt/zram/log.bind --upperdir=/opt/zram/zram2/upper
/usr/local/bin/zram-config: line 105: ./overlay: No such file or directory
ztab remove dir /zram1 /var/lib/openhab2 /openhab2.bind
/zram1
umount: /var/lib/openhab2 (overlay1) unmounted
overlay --lowerdir=/opt/zram/openhab2.bind --upperdir=/opt/zram/zram1/upper
/usr/local/bin/zram-config: line 105: ./overlay: No such file or directory
ztab remove swap /zram0 zram-config0
/dev/zram0 removed
removed '/usr/local/share/zram-config/zram-device-list.rev'
zram-config start 2019-08-10-11:53.....

This is a situation where shutdown -r was called, it’s stoping service, unmounting partitions, but they are not synced to disk and after reboot they are under root permissions again.

around line 105 in zram-config we have

mergeOverlay () {
103         echo "overlay --lowerdir=$ZDIR$BIND_DIR --upperdir=$ZDIR$ZRAM_DEV/upper" >>${ZLOG}
104         cd /usr/local/lib/zram-config/
105         ./overlay merge -l "${ZDIR}${BIND_DIR}" -u "${ZDIR}${ZRAM_DEV}/upper" >>${ZLOG} 2>&1 || return 1
106         sh -x *.sh  >>${ZLOG} 2>&1 || return 1
107         rm -v *.sh  >>${ZLOG} 2>&1 || return 1
108
109 }

and because ./overlay does not exist it can’t sync anything (if I understood what overlay merge does)
Checked all my instalations and there is no executable overlay in /usr/local/lib/zram-config/ which is being called for sync

as far as I can tell, it can’t work.

zram was installed by openhabian-config tool, am I missing something?

Yes, exactly. And we talk about its Motto “A home automation enthusiast doesn’t have to be a Linux enthusiast!”, right?
So first question - I need to move my OH server to another place. How can I shutdown it properly without going to SSH and typing obvious reboot command? Or if I need to add some HW? Now it seems I need to type special commands in Linux. Well… what about Motto?

Another thing is backup:
“And it has a backup tool included so no need to take out your SD card”. Well if I read this correctly, you have to be a Linux enthusiast to make it working. So again - this is wrong assumption for Openhabian user - I would rather take out SD card, clone it to another card on my windows machine, test it and store in dark cold place. This is a reliable and fool-proof way for non-linux enthusiast. So I need to reboot twice here too. So what about Motto here?

PLS don’t take me wrong. I’m an old user of OH since 2016 and 1.8, but it’s dissapointing to see, that Openhabian starts to require same skills to install and use, as standard OpenHAB and all requests to simplify it from newcomers are rejected from “Linux experts” as “You don’t need this”. You looking from different point.

For the most part, doing so means to stop (Windows) users from doing bad things (stupid? contraproductive? things to have impact they don’t understand ? choose your own term) that they know from their Windows world and believe to be a good idea while it is not on UNIX.
Such as to simply reboot when something went wrong. Such as to pull the power plug or to do the backup by taking out the SD card.
The proper answer in fact often IS “You don’t need this” or even “You mustn’t do this”. It protects you from making things worse. And no you don’t have to be an expert or enthusiast to setup Amanda.

That’s the bottom line. You’re not fair let alone motivating to continue our efforts. We ARE trying to simplify things but that’s only possible to some extent and it can never become idiot-proof.

interestingly it won’t create proper instalation of zram at FIRST run
I had to do

  • sudo service zram-config stop
  • sudo openhabian-config
    ** System Settings → Use zram (to reinstall it)

then overlay executable appeared
then it synced zram to disk before reboot, cripled permission are still there after reboot, but it has got content before reboot

So for anyone using zram, make sure you have overlay in /usr/local/lib/zram-config/ if not, perform zram-config stop & reinstall zram from openhabian config tool again

By stopping zram-config service while you dont have overlay in place you will likely going to loose everything you’ve done in OH!

1 Like

Thanks for finding out. Interesting indeed as it’s always the same code that gets called to install zram.
On first, second and all subsequent runs :thinking:

Is it really ovelay thus a typo in the code or is that just a typo here of yours ?