there is very little space in the ZRAM file system, and for good reason as every MB in the ZRAM file system is one MB that is no longer available to running programs as RAM.
InfluxDB databases can become quite sizable
ZRAM does not automatically flush the data back to disk except on shutdown; should your machine crash without shutting down nicely you will lose all the data saved since the last nice shutdown
So I recommend not trying to run InfluxDB or any other database out of zram. And once you attach an SSD the whole problem that ZRAM is attempting to solve, namely SD card wear out, goes away if you move the writes to the SSD, I’d recommend focusing on moving as much as possible to the SSD instead of trying to move everything to ZRAM.
I talks to InfluxDB through InfluxDB’s API. It won’t know and won’t care where InfluxDB is saving it’s data.
Thanks Rich. As I had already bought the SSD, I was leaning that way and use the ‘move to USB’ option.
I need to dig around the forums more though as running the option on openhabian has stopped the Pi4 booting completly. I thought the Pi4, with SD card in, should still boot from SD and run all the root files from USB(SSD). I have tried copying the latest .elf and .dat files onto the SD but its not booting.
Put another sd card in card reader into PI to mirror for failure.
Mount the SSD and get stuff that writes alot to move to SSD.
Do automated backup to SSD as well.
You probably already have all the hardware needed to achieve this or its cheap to do. This way if you have a failure you have multiple ways to recover.
As I am struggling in trying to get my option 2 to work with the Pi4, I think I’ll follow all of your inputs and try option 1.
It’ll keep openhabian running as is with zram. InfluxDB files can then be held on the SSD, not needing to worry about the zram impact. I know Markus Storm is trying to get influxDB pre-configured into the persistance (zram’d) directory and if that happens in the future, I may revist that option.
You can see in the little table that a single local attached SSD will do a lot more than a normal openHab install would need. If you have a lot of noise being recorded and a lot of concurrent queries and it became an issue you could improve performance by putting WAL and DATA on different SSD but I would be amazed if you needed too. I am sure if there was that much noise it would be causing issues with performance elsewhere.
Just to follow up, I’ve left everything on SD with zram but configured influxdb to store data to the SSD. I have copied a backup of my old influxdb database over to the new setup and merged the old data with the current new.
All seems to be working well. I may even write up the procedure I went thought if anyone thinks it’s of any use?
Hello,
I came across your post while trying to do the same. I work with a Raspi4. I too tried moving the root to an SSD, but it failed. Without SD card it would not boot, with it it boots, but I can not run OpenHAB:8080 as it does not connect.
So I like to move Influxdb writes and possibly the log file writes to a SSD drive. I can see that the sprj4 db is still active and can not be disabled. So in order to kill ZRAM I would have to move that to the ssd too, or leave ZRAM on (and save for an 8Gb Raspi…). How did you do that? I have no clue about Linux…
I think the Pi4s have an issue with certain USB adapters/chipsets and without a compatible one, you won’t get the Pi to boot from an SSD. Firmware updates may have fixed this but I haven’t checked.
I’ve been running openhabian and openhab 3 with zram off the SD and the influxdb on an SSD since this post and it’s all been fine. I can put up an idea of the procedure I went with but it’s not a complete instruction set and my Linux/command line knowledge is based on forum research and trial and error.
Frankly, if that’s the case then you shouldn’t be messing with the system. As I already told you on another thread SSDs and OS modifications are not supported from the openHABian side.
I suggest you just drop the idea to use InfluxDB and stay with the OH3 default rrd4j database. It’s fixed size and will work within ZRAM.
As long as you have backups and are happy to “play and learn”, the following is roughly what I did. If anyone can see any glaring errors or things that I should or shouldn’t have done, please correct me.
Check the ‘sd*’ dev for the SSD.
blkid /dev/sd*
(may need to use SUDO in front) e.g my SSD was ‘sda’. Note down the UUID of this device. E.G. 988555as-cac-*********
Format SSD sudo mkfs.ext4 /dev/sda
(If not already done).
Make dirs (This folder is given ‘root’ permissions) sudo mkdir /external/ssd sudo chmod -R 777 /external/ssd sudo mount /dev/sda /external/ssd
Check Folder is created ls /external/ssd
Set-up fstab so sda doesn’t change on re-boot sudo cp /etc/fstab /etc/fstab.backup sudo nano /etc/fstab
Edit the fstab
Edit file by adding the following. The UUID is the string of numbers in the first step. UUID=988555as-cac-********* /external/ssd ext4 defaults,nofail 0 0
Re-boot sudo reboot
BEFORE CHANGING INFLUXDB.CONF
Make new influxdb folders sudo mkdir /external/ssd/influxdb sudo mkdir /external/ssd/influxdb/meta sudo mkdir /external/ssd/influxdb/data sudo mkdir /external/ssd/influxdb/wal
Copy current data to new folders sudo cp -rp /var/lib/influxdb /external/ssd
(I can’t remember if I needed: sudo chmod -R 777 /external/ssd/influxdb first?)
Edit ‘influxdb.conf’ sudo nano /etc/influxdb/influxdb.conf
Edit the three directories in the ‘influxdb.conf’ file and save.
Change new folder permissions to allow ‘influxdb’ sudo chown -R influxdb:influxdb /external/ssd/influxdb
Restart ‘influxdb’ systemctl restart influxdb.service systemctl status influxdb.service
I can see that OH´s main purpose is automatization with no need to store a lot of data with high resolution for a longer time.
I probably misuse OH.
I have a need to gather the data of my 6 PV rectifiers and OH with its bindings works well for that. That data comes in every 5 sec, I can reduce the amount of course. Should I see a failing equipment, then it is good to be able to go back in in time and check. Doing synced cross-calculations works better with high resolution data for time alignment problems.
And I have20+ thermostats with their temperature sensors. I can happily control them via my Fritz box(es), but I can not visualize many rooms together vs outside temperature in order to change settings. And again, OH is great for visualisation, and there too I like too keep the data and do a bit more with it eventually.
So, yes, ZRAM is great, but the DB it holds in ram is tiny, and even USP fail. And I feel better having that data in a proper file system recorded by a non-data reducing software.
I am kind of afraid that my problems may originate from the USB3 vs USB2 problem the Raspi4 has and I will do some check later, the problem could very well be outside of the OS.
Once setup, I’ve got “/backup/influxdb” and “/backup/openhab” folders on the SSD. I have the following ‘backup.sh’ bash file in the “/backup” folder and occasionally run the bash file. It makes a copy of the influxdb database as well as (because I wanted to but don’t need to) running the openhab backup routine and copies it to the SSD.
I attached my USB3 on the Raspi‘s USB2 port and have moved the root to that USB using step 39 in the configuration menu. That boots (with the SD card still inserted) and seems to work. I had tried that previously but had attached that Ssd to an USB3 port, not knowing about the USB3 problem that the Raspi still has with some drives.
As said it works well. I have disabled Zram. Is there any write process still left for the SD? Is the Influxdb‘s size now only limited to the ssd‘s size?
I guess I have a problem which is covered under “no support”. Nevertheless: I have used the Openhabian utility and moved Oh to an attached SSD, still leaving the SD in the slot of the Raspi4 and booting from it. That works great, I can verify that OH runs off the USB-SSD. Now I like to make a copy of the SD to have a backup for booting if the original one should fail. Now, I tried to use SD raw copy as found in Openhabian-config to copy that SD to another one, but it does not work. I can not boot from that new SD. Somewhere in the boot process it wants to set a “kill switch” or whatever and hangs. I tried Etcher on a PC and it does not work I have two USB-SD adapters and try to make a straight clone. It works great when running OH from an SD card alone and cloning that. But cloning the boot SD of a SD/SSD system does not work - where is the problem?