After migration to OH3 I have problems with influxDB on openhabian.
All data is collected and I can view measurements in Grapfana. If I restart my raspberry with sudo reboot data can be viewed also after reboot, but if I do emergency shutdown (disconnect raspberry power supply) all data between last reboot and emergency shutdown is lost.
I lost 3 weeks of data and I’m pretty sure that it was because of power loss. Currently I tested it manually by disconnecting power supply 6h after last sudo reboot and as I excepted all data in this timeframe was lost.
Any idea why it is so, or how to manually force influx to store data for example once per hour ?
It is very dangerous to just pull the power on any computer running off of flash memory. If it happens to be writing to the card when power is lost, you corrupt not only that file but any parts of files that are sharing the same sector as what was being written. With wear leveling that could be anything, other OH configs, the kernel, your database, etc.
That’s probably not what’s happening here but it’s an important warning. You should never pull the power from any computer running on flash based memory unless there is no other option. And you should be prepared to restore from a backup or rebuild the machine after having done so.
Since you are running openHABian I suspect that you are running with ZRAM enabled. That causes most of the writes to be made to a RAM disk which greatly saves wear on the SD card and making the machine more stable over all (it’s also way faster). But the last time I paid attention to it, ZRAM did not support periodic dumps back to the SD card. It only does so when ZRAM itself shuts down cleanly. When you pulled the power zram was unable to shutdown cleanly and anything written to ZRAM was lost.
Your first priority should be to figure out why pulling the power was necessary in the first place and see if you can prevent that in the future. If you can’t or still want to have everything written to disk, unless and until ZRAM supports periodically dumping the RAM disk you’ll have to disable ZRAM and have everything written to the disk. However, realize that it was likely the ZRAM configuration that helped avoid file system corruption when you pulled the card in the first place.
Thank you for your replay .
I know how dangerous is to cut off power supply from raspberry. Unfortunately in the place where I live power outages happen, more an more rarely but still. It wouldn’t be a problem for me if i lost even a week of data in that case, but as I wrote last time I lost 3 weeks.
Currently the only way I see to store my data is to write script that would be periodically reboot my raspberry, but this is not the cleanest solution or build in UPS HAT.
What is weird, I’m pretty sure that I had zram enabled before migration to openhab 3 and in 3 years of use I haven’t noticed any such data loss, although power outages have happened.
Edit:
Like I wrote it is not the cleanest solution but maybe this will be useful for someone. I used corn to schedule reboot.
Start corn jobs editor:
sudo crontab -e
If it is first time you will get:
Select an editor. To change later, run 'select-editor'.
1. /bin/nano <---- easiest
2. /usr/bin/vim.basic
3. /usr/bin/vim.tiny
4. /bin/ed
Choose 1-4 [1]:
Choose 1 for nano editor
Then add reboot schedule ( in this case each day at 0:30 ):
30 0 * * * /sbin/shutdown -r now
ctrl + s - for save
ctrl + x - for exit
This should restart raspberry every day on 00:30 limiting possible data loss to one day.
As OH changes over time from version to version, so to does openHABian. The way openHABian uses and works with ZRAM has definitely changed over the last three years.
You shouldn’t have to reboot the whole machine. I think it would be enough to simply restart the zram service.
Influx data storage was moved to ZRAM only some months ago.
Reading all of this I wonder why you don’t run daily backups ? Setup Amanda.
This explains a lot.
Problem here is that recovery must be done manualy. If power loss happens I would like the data collection to influxdb to continue automatically. As far as i understand with Amanda influx data would be recovered to last backup state. This means that all data between the power failure and the data recovery procedure will be overwritten with the backup unless I manually merge the database.
Am I right or wrong on this point?
I’ll have to try this solution out and see if there are any negative effects on the overall system. A reboot so far seems to be the safest solution.
That’s right, but it’ll require to also shutdown everything that runs on ZRAM i.e. openHAB and InfluxDB.
That’s right but ultimately you cannot have it all.
Get an UPS, that should basically do.
The current openHABian setup is probably the best balanced one to fit most user’s needs.
If to you (unlike to others) historic data is so important then you should choose a different setup, you could also add another medium (second SD or SSD) and change the Influx config to store data there.
1 Like
I understand that for most users this is the best configuration. My questions are not a sign of malcontent but lack of knowledge how to configure this system to meet my specific needs.
I know, I’m data addict 
. I’ve already managed to kill a couple of SD cards and have now put openhabian on an RPi 4 with SSD. In that case, does it make sense to use ZRAM for influxDB?
No. Reconfigure InfluxDB to have its data stored in a directory on the SSD, then.
(no I don’t know how off the top of my head that works but I know there’s a config parameter for that so if you look for it you should manage to accomplish that)
Or did you move the whole system from SD to SSD ? In that case you should uninstall zram altogether.
My current system is a fresh installation of openhabian, directly on the SSD, so in this case uninstalling the ZRAM will be the solution for me.
Thank you both for your help.