Basic setup for persistence

Add-ons Persistence Services
1

I try to set up a persistence layer but somehow the tutorials are missing some details and I honestly don’t know what (and where) to check.

First of all I have configured my persistence service and querying it via the API-Explorer shows me for rrd4j the following response:

{
“serviceId”: “rrd4j”,
“configs”: [
{
“items”: [
“*”
],
“strategies”: [
“restoreOnStartup”,
“everyChange”,
“everyMinute”
],
“filters”: [ ]
}
],
“aliases”: {},
“defaults”: [
“restoreOnStartup”,
“everyChange”,
“everyMinute”
],
“cronStrategies”: [
{
“name”: “everyMinute”,
“cronExpression”: “0 * * * * ?”
}
],
“thresholdFilters”: [ ],
“timeFilters”: [ ],
“equalsFilters”: [ ],
“includeFilters”: [ ],
“editable”: true
}

But now I want to persist e.g. the temperature of one sensor. How to persist the states of this Item? AND how to to check if the values are persisted?

If I’m successful with this kind of setup the next question would be how to transfer data from rrd4j into a DB like MapDB or InfluxDB?

I’m expecting that Graphs from the DB is more easier especially for historic values.

2

Note, please use code fences when posting code like that. Unfortunately the “preformatted text” icon in the editor is really only intended for embedded formatted text like this. For multi line you need to use one of the icons on the right, or just type it in:

```
code goes here
```

It’s already being done. That configuration is saving every change to all Items rrdj4 supports and it’s saving every minute (which is an rrd4j requirement) with restoreOnStartup (meaning that when OH comes up it will set the state of the Items to the last recorded value so it has a meaningful value instead of NULL.

There are lots of ways, but the easiest is to go to MainUI → Settings → Items → Some Number Item and the default widget for the Item should have a sparkline behind the current state.

image
image907×304 13.5 KB

If you have a semantic model configured, you can click the “Analyze” or “Analyze All” option on any widget to see a chart.

From MainUI → Developer Tools → API Explorer → Get /persistence/items/{item name} you can query for all the values between two date/time stamps.

Since you don’t really have this set up yet, you probably don’t have any real data to transfer. But OH can write to all three at the same time. You can have as many persistence engines set up as necessary. I recommend modifying the default strategies based on what you are using each database for. For example, MapDB only saves one value so it’s best for restoreOnStartup. rrd4j is good in that it doesn’t grow so the everyChange and everyMinute strategy is reasonable for all supported Items. External databases like InfluxDB you will probably want to pick and choose which Items get saved to that.

You set the persistence strategies in MainUI → Settings → Persistence.

3

Thx for the response and helping me to clarify some basic settings. Also thanks for the hint with fences. With the suggested checks I see that the data are stored and I get somehow a chart. Details about this I like to ask somewhere else.

I reviewed my settings and see that I have configured three persistence layers

  1. InMemory
  2. rrd4j
  3. MapDB

I see that I can query details for an item with the MapDB. InMemory and rrd4j won’t have listed any items. The API explorer returns an empty array. Which leaves me with the question if I have something wrongly configured or “everything is fine”. I am little bit unclear about the consequences for the one or the other.

Looking into the documentation I am left confused even more. The documentation for MapDB says its a complex install - which I haven’t perceived. I don’t understand why I need influxDB because it seems running without it.

To my understanding rrd4j is great for time series since DB won’t grow but MapDB has a growing behaviour. I think I won’t mind if the data is growing since the storage shouldn’t be THAT much expensive but the SD card would potentially wear off.

Nevertheless this raises a basic question which I couldn’t find in the documentation: Where is the data stored (by default)? Is it on the SD-card? Is there some “basic” example which would transfer regular stored states e.g every hour into a DB?(either attached as USB/HDD or remotely)

Rem: Don’t know if I’m running off topic but I didn’t find these information in the documentation

4

InMemory is usually only used for future predictions like weather forecasts or electricity prices. If you are not doing any of that sort of thing you may not need it.

What’s your query? Also remember that rrd4j only works with Number, Switch, and Contact Items.

I did try http://openhab-hostname:8080/rest/persistence/items?serviceId=rrd4j just now, and it returned an empty array, which is not right. There might be a bug that needs to be addressed. But if you see a chart, data is being saved. And a query on an individual Item returns results.

What’s your default persistence? That’s where charts will pull their data by default.

I don’t see anything complex about MapDB’s documentation. It really doesn’t even say anything at all about installation. Like any other database add-on, you need to install it from the Add-on store and optionally configure it’s persistence strategy.

What made you think you needed InfluxDB in the first place? Technically you don’t need any persistence at all. If you want restoreOnStartup, MapDB is the best choice for that because it only saves one value and it supports all Item types. If you just want some charts of your data, and you don’t mind the data becoming compressed/decimated as it gets older, rrd4j is your best choice. InfluxDB et al is really only needed if you want to do some analysis of your data outside of OH or want to use a third party tool to generate charts, like Graphana.

No, MapDB only saves one value per Item. Period. Both MapDB and rrd4j have a fixed size per Item and never grow (except if you add new Items). From the docs you linked to above:

The MapDB (opens new window)persistence service is based on a simple key-value store that only saves the last value.

If you are using openHABian with zram enabled (which is the default), rrd4j and mapdb gets written to RAM and only written to disk during a graceful shutdown of zram (which usually means a reboot).

It depends on the database. If you are using a third party DB like InfluxDB, it’s where ever and however you set that up.

If you are using an embedded database like mapdb, rrd4j, sqlite, etc., the data is stored in $OH_USERDATA/persistence (e.g. /var/lib/openhab/persistence).

InMemory doesn’t get written to the file system ever. It only exists in RAM.

5

Thank you for clarifications. I get a better idea how things might work - or at least are supposed to work.

The API-Explorer seems weird. I get a list for Items, but I cannot query it for rrd4j like


https://home.myopenhab.org/rest/persistence/items/TemparaturSensor1_Temperature?serviceId=rr4dj

which results in

{
  "error": {
    "message": "Persistence service not queryable: rr4dj",
    "http-code": 400
  }
}

For inmemory I get

   "name": "TemparaturSensor1_Temperature",   "datapoints": "0",   "data": [] }

while for mapdb I get only ONE datapoint which is the same as you’ve described. When I do NOT limit the query to a service I get 1440 data points. Which I would guess are from rrd4j - right?

If I understood correctly then I need MapDB only for persitence of a single value for start up. All the others are gone - correct?

The DB are written when I “gracefully” shutdown. When I just turn the Pi off the data is lost as well. Out of curiosity: Is there a “simple” widget which would persist the data when clicking on a text?

6

Depends on what you set as the default but if you’ve only these three installed then it must be coming from rrd4j.

You’ve spelled the service wrong. It’s rrd4j, not rr4dj.

And you run the risk of corrupting your file system. If the RPi happens to be writing out to the SD card when power is lost you lose not only the file being written but anything that’s also in the same sector. With wear leveling, that could be any file or part of the file system. You should never just yank the power from an SBC running on flash memory.

You could write a rule to do that which gets called from a widget. But widgets don’t do anything themselves. They just send commands to the OH server.

7

Thanks for highlighting the typo as well as for other clarifications.

Some other questions I am looking for clarifications but I’ll use different threads.