I have read several topics on this board now regarding the reduction of write operations on the SD card.
I would like to reduce the write operations after the start of openhabian to a minimum, but I don’t want to loose configuration edited for openhab2 in the meantime.
What I have leant so far:
Make /var/log a tmpfs, that should be no problem. Loosing logs is acceptable.
You could also make any directory tmpfs, for example, where databases are stored and written regularly. Then you could use rsync in a cronjob i.e. to copy modifications to a “persistent” directory somewhere.
So far, with a standard distribution of openhabian, the first point alone should also be sufficient.
Now, I would like to add persistence and don’t really know, which one to choose. My setup mostly contains sensors and switches that use MQTT and I would like to store i.e. the sensor values.
I guess, I should use RRD4J or MQTT persistence.
And that leads to my next question:
Where is the data stored an how often are they written? Would it be possible to go for point 2 here? Or is there a better solution?
I don’t want to use additional USB Sticks or HDD. I am working on a backup concept to store nighly backups of important directories remotely on an owncloud server. I just want to extend the lifetime of the SD card!
Did anyone allready do something similar or can answer the question, which directories are most regularly written on openhabian?
openHABian 1.3 comes with Amanda, a backup system. Try that one, it provides backups to AWS, too.
Important directories to backup are already set there.
Yes, I have seen this topic and also took a look at the backup possibilities in openhabian 1.3.
I don’t want to put my backup on a amazon service, I prefer my encrypted owncloud!
But Amanda can backup to any local location, so after that, I can just put it on any server, I guess.
But the more important prt, in terms of getting-to-know, is the write-saving.
Is there any good approach?
Is there a good way or tool to count write operations? I am currently using lsof, but I guess that do not work for i.e. database files that are not opened all the time, does it?
Just to head you off from going down the wrong path, I’m about 80% certain that the MQTT persistence is not queryable. You can write to it but restoreOnStartup and charting is not possible.
Another alternative beyond what is discussed in the thread Markus pointed you to is to host the persistence database on some other machine. Many of the databases are separate serverices which are available over the network. So you could for example host InfluxDB on your OwnCloud server and have OH write to it over the network.
You are right.
Comparing the different possibilities for persistence, I realized that.
So the only practical options are
rrd4j, which would be okay, I guess, since I probably don’t want to view sensor data for a year ago in every detail. The advantage would be a nearly fixed database size
db4o, wich would keep every sensor value as configured, but of course grows with time, so I would run into problems at some point
InfluxDB seems to be a good and simple option as well, but also grows with time
The option to host the persistent database on another host depends on the setup. My owncloud ist hosted in the web, so it is not very practical to keep persistence there. So I need to have persistence locally on the Pi.
That leads back to the main problem: The write operations caused by the persistence. I guess it is not practical to have an tmpfs for the database as it could have some bigger size and the memory is limited and needed by openhab2…
Perhaps the persistence services can be configured to only commit changes to the DB every … minutes?
Again: see thread.
Basically, what you’re trying to do is not really worth the effort.
If you truely want to reduce writes, get a USB stick or NAS and put the log and persistence DB there.
I thought db4o was dropped in OH 2. SQLite is roughly equivalent though.
Actually you can set a retention policy with InfluxDB that will automatically delete values as they get beyond a certain age. Also, in practical terms you are looking at 1-5 mb of data per item per year. One I did this math I stopped worrying about the growth of the DB.
You also have the option of MongoDB and any JDBC compliant database (MySQL, Oracle, postgres, SQLServet, etc.)
Unlikely but anything can be possible. I can think of cars where one might want to do that for performance but usually the main requirement of a database is to never lose data ever and such a configuration would break that.
I think your best bet would be to get an external sdd or hdd and move the db files to that. I’ve seen 350 gig drives made for the Pics for $30.
