From RRD4J to an other DB

Add-ons Persistence Services
1

I have been using OpenHAB since the first version. I have never paid much attention to the persistence service and therefore I installed rrd4j by default.
Since there are often incorrect values in the graphs (for example, internal temp of -700° or +100° Celsius, or in example energy meter of -1000000 kW), I would have possibility to eliminate those data with simple commands. However, I understand that this is not possible with RRD4J.
I’d like to change persistence service since i don’t have space issues and I prefer to have a very large database, even a few GB, with all the data modifiable as I wish, rather than what happens with RRD4J, which has a predefined size. Is it possible to export data from RRD4J to some other database without losing the historical data? I have tried exporting the values to CSV, for example, but with rrdtool → ERROR: ‘HEM_E1.rrd’ is not an RRD file
(or even with “org.rrd4j.inspector.RrdInspector”) it seems that the rrd files present in /var/lib/openhab/persistence/ are corrupted/not readable.
Thank you for any suggestions!

2

You’re laying out an X-Y type of problem.
If you have recorded bad values, a database change won’t change anything about that, and ‘post-sanitizing’ recorded data just isn’t the right solution to the problem.
I rather suggest a different approach: you need to tackle the problem at the source, i.e. ensure you only persist validated data. That may not be simple but there have been quite a number of recent enhancements such as persistence filters that will help you with that.
Thing is, if you don’t do that properly, you will keep encountering the same problem over and over again, no matter what database you switch to.

3

You can get access via the openHAB REST API,

http://192.168.178.22:8080/rest/persistence/items/<itemname>?serviceId=rrd4j&starttime=2023-12-31T00%3A00%3A00.000Z
but of course you have to set an authentication bearer in the header.
I’m not sure how to ensure to get all persisted data in one chunk, as the maximum time span may be restricted.
Anyway, the result is a JSON object:

{
  "name": "<itemname",
  "datapoints": "10312",
  "data": [
    {
      "time": 1703667600000,
      "state": "12251.418710909093"
    },
    {
      "time": 1703671200000,
      "state": "12251.503377166657"
    },
...
   ]
}

time is a unix timestamp, so depending on the target you will have to calculate the new timestamp…

4

That’s correct, however, the problem with old data persists, until the data is shifted out of the rrd file.

5

But it’s cosmetic at best.
You can look after it once you have fixed the problem at the source, starting with “just ignore it” there’s several options. But none of them makes sense for now.

6

There have been some scripts posted to the forum here and there which use the OH REST API to pull data and republish them to move the data from one database to another. I think InfluxDB to RRD4J were used but they should be modifyable to work with any persistence supported by OH.

You’ll have to modify those to filter out the erroneous values of course.

But I completely agree with @mstormi, first you need to make sure these values don’t get saved to the database in the first place using persistence filters and the Basic Profiles. And once you’ve done that, you can assess whether you really need to move to another database for other reasons or not.

7

Thanks everyone for the replies!
I expressed myself badly, i’d like change the database anyway also for completeness of data.
But it seems to me that I understood that the data present in the persistences cannot be saved, is this correct?

8

No.