I am trying to establish a good persistence solution for my openHAB setup. After playing around with various services for a while I have come to the realization that there is no “one service that fits all”, so I am basically setting up a mixed service environment.
As an example I am using the mapdb service for all items that I want to “restoreOnStartup”, the my.openHAB service for all items that I want to access from IFTTT, etc.
Right now I am looking for a good service to persist data that I want to graph. My problem is that I would like to have a fairly high frequency of samples (like an everyChange, everyMinute strategy) but at the same time I do not want to let the datatstore grow to fill my entire disk.
I started out by using RRD4J but after struggling a bit with this I realized (with some help from other kind souls on the forum) that this service/concept is not really storing the true value but instead stores the values as “an average over some time interval”. Given that I want the true values to be stored this will not do.
Clearly, since I do not want to use an infinite amount of diskspace, I cannot expect to have a very long history of data, but that is OK. So what I am looking for is a persistence service that can store the true values (no average, or other clever tweaks) but only for a fixed amount of time (i.e. a finite history), e.g. 24 hours, 7 days, or something similar.
After the fixed amount of time, the oldest data is simply thrown away.
I am no InfluxDB expert but I use it for persistence and it is very good, especially when you chuck Grafana on top for rendering graphs etc.
I don’t do this, but I believe you can setup InfluxDB so data expires after a certain period. As I say, I am no expert but I think if you have a read of the docs you should be able to work it out.
NOTE: I think you still have to use InfluxDB v8, since the openHAB persistence binding hasn’t been updated to work with v9.
Actually, InfluxDB used to be my persistence service of choice. I was very happy with the InfluxDB + Grafana combo - someting I set up inspired by one of your earlier posts on the topic,
Lately, however, I have moved to a Raspberry Pi, and InfluxDB apparently is not readily available on this platform, I am still hopefull that it will eventually be, though…
Anyway, while using InfluxDB I never looked at the possibility of letting data expire. I will certainly have a look at this to prepare myself for “the day that InfluxDB comes to my Raspberry”…
What about a periodically time-triggered rule that would delete data older than a certain age? The rule would directly access the data store implementation since the PersistenceExtensions don’t support deleting data. This would be easy to do with MySQL and some SQL delete statements.
Yes, I think this would be a possibility. I am pretty far from being an MySQL expert, but I think it would even be possible to setup such a “housekeeping job” as an automatic job “inside” the MySQL server? I’ll look into this and see if it can work out for my purpose.
Ideally, though, I would like an embedded database solution (no DB server) and I would have liked the concept of data expiry to be part of the persistence configuration in openHAB. So, basically for each item I could define a persistence strategy that included the maximum history of the item, e.g. store “everyMinute”, keep “last 7 days”, etc.
I am still hopefull that it will eventually be, though…