Most of the time it works well.
Today I came across a situation I never saw before:
I could not SSH
I could not use logviewer
I could not use Paper UI
I could only use Basic UI
The rules continued working
I took the power out and back in to restart - should I do something else?
After restart:
syslog has no entry for ~3 hours before reset.
openHAB.log has many NULL text in a single row before the restart, and no entry for the ~3 hours.
What may cause this?
How can I make sure it will not happen again?
You didn’t have much of a choice here but you should only remove power from an RPi as a last resort. When you do so you run a very high risk of corrupting your file system and completely breaking your system.
It’s impossible to say because you’ve provided no details beyond “today it stopped working” what went wrong. The usual culprits are:
SD card is full
File system became corrupted because of a power loss
Apologies for not mentioning that I use a SSD HD, not SD card.
The SSD HD is connected to a powered USB HUB (along with the RPi), in order to make sure it is always powered when the RPi restarts (I came to the conclusion that this was the reason for the issues I had rebooting my RPi).
Also, when I shutdown / reboot my RPi (through SSH or cron) should I first stop the openhab2 (or any other) service?
I have tried to clear the cache, but it happened again. This time the logviewer reported Internal Server Error and there was no SSH, but Paper UI (and even the OpenHAB mobile app) worked correctly.
I noticed that every time it happened, the last record on syslog was related to influxdb:
May 14 16:17:24 openHABianPi influxd[510]: ts=2019-05-14T13:17:24.665185Z lvl=info msg="Reading file" log_id=0FPcgF9l000 engine=tsm1 service=cacheloader path=/var/lib/influxdb/wal/_internal/monitor/110/_00014.wal size=10509790
May 14 16:17:32 openHABianPi influxd[510]: ts=2019-05-14T13:17:32.165719Z lvl=info msg="Reading file" log_id=0FPcgF9l000 engine=tsm1 service=cacheloader path=/var/lib/influxdb/wal/_internal/monitor/110/_00015.wal size=10511984
May 14 20:03:28 openHABianPi systemd[1]: Time has been changed
May 14 20:03:28 openHABianPi systemd-timesyncd[285]: Synchronized to time server 192.248.1.162:123 (2.debian.pool.ntp.org).
May 14 20:03:28 openHABianPi systemd[1]: apt-daily-upgrade.timer: Adding 42min 8.561412s random time.
May 14 20:03:28 openHABianPi systemd[1]: apt-daily.timer: Adding 7h 49min 40.745260s random time.
May 15 08:34:22 openHABianPi influxd[520]: ts=2019-05-15T05:34:22.005224Z lvl=info msg="Reading file" log_id=0FQV_wO0000 engine=tsm1 service=cacheloader path=/var/lib/influxdb/wal/_internal/monitor/111/_00006.wal size=10503155
May 15 08:34:28 openHABianPi influxd[520]: ts=2019-05-15T05:34:28.950976Z lvl=info msg="Reading file" log_id=0FQV_wO0000 engine=tsm1 service=cacheloader path=/var/lib/influxdb/wal/_internal/monitor/111/_00007.wal size=6860911
May 15 16:07:01 openHABianPi systemd-timesyncd[271]: Synchronized to time server 210.173.160.27:123 (2.debian.pool.ntp.org).
May 15 16:07:01 openHABianPi systemd[1]: Time has been changed
May 15 16:07:01 openHABianPi systemd[1]: apt-daily-upgrade.timer: Adding 5min 7.568644s random time.
May 15 16:07:01 openHABianPi systemd[1]: Starting Daily apt download activities...
May 15 21:48:31 openHABianPi influxd[519]: ts=2019-05-15T18:48:31.711544Z lvl=info msg="Reading file" log_id=0FRD0Zpl000 engine=tsm1 service=cacheloader path=/var/lib/influxdb/wal/_internal/monitor/111/_00014.wal size=10510620
May 16 14:06:03 openHABianPi systemd[1]: Time has been changed
May 16 14:06:03 openHABianPi systemd-timesyncd[291]: Synchronized to time server 203.217.204.135:123 (2.debian.pool.ntp.org).
Does it mean that the issue is related to influxdb?
How do you suggest me to handle it?
Should I assume that the file system is corrupted, as @rlkoshak suggested, and go for installing OpenHabian & OpenHAB from the start?
What would be the best / safest / easiest way to re-install, while maintaining as much as possible from my existing configuration and rules?
Should I reformat my SSD HD, reinstall and then restore my configuration using the latest backup I have?
Is it safe enough, to make sure I am not putting back the cause of the issue, by restoring a backup?
Will this re-install all the add-ons / bindings / services?
Will this keep all the “channel names” (including the Z-Wave nodes), or will I have to re-configure all items?
Is there a different process I should follow?
Is there a specific file / folder / anything I should keep before reformatting my SSD HD?
My experience when thing like this happen a restore of backup after restoring the os has shown no issues.
I have had multiple pi systems get corrupt like this. I just rebuild os and restore and all is well. I believe the reason behind this is because most of your config files are read only so they are less likely to be corrupt.
In all my cases I don’t believ my sd card was bad only corrupt from some type of power glitch.
