What is the priority of persistence strategies when multiple apply, i.e. *

Using rrd4j I want to define a fine strategy for chart items, and a basic strategy for other items. Below is an illustration with overlap between all three lines.

Since the gChartTemperature group is also part of all items (*), I’m wondering what determines which strategy applies to them. Is the order of strategies relevant, or will named group items and other items automatically override all items (*) with less specificity?

And how about if you specify both a group and an item that is part of that same group, how to ensure that the item strategy overrides the group?

Strategies {
    everyMinute : "0 * * * * ?"
}

Items {
    * : strategy = everyChange
    gChartTemperature* : strategy = everyMinute
    RoomTemperature : strategy = everyMinute, everyChange
}

This information is not found in the docs.

It’s additive. The * strategy applies to all Items. The other lines will add additional strategies to those Items but do not override the * strategy. So in this example, RoomTemperature would have everyChange and everyMinute even if it were not listed on its own line. But if you had RoomTemperature : strategy = restoreOnStartup then RoomTemperature would be everyChange, everyMinute, and restoreOnStartup.

Note: everyMinute is required for all Items stored in rrd4j whether or not you intend to chart them.

Thanks! What you just said would be useful to others in the docs, for sure.

No, since openHAB2 this isn’t the case anymore. Required is a verbose sample rate which is taken care of by the default archive configs!
ReConfirmed by a test today!

So what is required? The docs for the add-on state

This is accomplished by saving a fixed amount of datapoints and by doing data compression, … So while you might store a sample value every minute for the last 8 hours, you might store the average per day for the last year.

If everyMinute isn’t required what periodicidy is required?

And shouldn’t the default strategy use that instead?

default persistence strategy which persists every Item on every state change and at least once a minute.

IMHO none!
The periodicidy (never used that english word :face_with_peeking_eye:) is done via the sample rate. The default configurations ( i.e. Datasources) of rrd4j (default_numeric, default_quantifiable and default_other) do use a sample rate of 10 sec or less. That way a value is put into the database every 10 sec (or 5 sec for default_other), to be more precise those values are put into archive 1.
The used “strategy” of openHAB defines at which time the actual state of an item is provided to be used by the connected persistence service. How often this state is “used” (stored) by the database depends on the used persistence service and its configuration.
One could setup an “everyHour” strategy, even though the actual state of the item might be changing during the 60 minute period, the connected persistence service would only store the unchanged value.

At least this is my humble understanding of the behaviour of rrd4j!

IMHO using “everyChange” would be enough, however I would have used “could” instead of “should”.

I think there are some remaining “spots” in the docs where the old requirements of rrd4j are still in place or hab been in mind while writing a default setup. I’d say they were simply missed.

True but in this case it’s the default behavior of the add-on itself. Given a significant portion of OH users are running on SD cards, if rrd4j doesn’t need to write a record every minute for every Item that would be a huge savings in SD card writes.

Or am I misunderstanding and rrd4j is going to create a record independently of what OH does so the writes are happening anyway?

Yes, for example my test-item with an “everyHour” strategy got records in archive one every 10 seconds.

Your argumentation about unneccessary SD-card writes should be pointed toward the datasource (default) setup. However, I think this change was done on purpose by the author.

BTW: I wrote a comment on your documentation PR, stating that I would suggest to delete the complete sentence.

Yes, I saw that comment and eagerly await your test results.

But beyond that one sentence in the docs I’m concerned about the default strategy employed by the add-on. It seems like the the everyMinute isn’t necessary and it should just be everyChange.

Lets continue in here for the moment.

The test shows that changes are recorded ( changes of .1 degree Celcius), so it seems to be working.
I do understand your concerns about the default strategy, however what would change when using “everyChange” only? rrd4j would continue to write records at every 10 seconds.

It would normalize the default strategies across all the persistence add-ons. It would prevent us old timers from assuming that everyMinute is still required.

In the docs we do need to capture that a record is recorded every 20 seconds because that impacts whether and how many of the Persistence actions work (e.g. lastUpdate). If a user is assuming that values are saved every minute but rrd4j is saving every 20 seconds lastUpdate will never return a timestamp more than 20 seconds ago and confusion will result.

Out of curiosity I queried one of my Items that updates periodically but definitely longer than every 20 seconds.

I’m getting records every 60 seconds, which is what I would expect with an everyMinute strategy. Does the REST API not return the records that rrd4j writes itself inbetween entries?

Now I’m really confused.

    {
      "time": 1675357860000,
      "state": "42.0"
    },
    {
      "time": 1675357920000,
      "state": "41.050000000000004"
    },
    {
      "time": 1675357980000,
      "state": "41.0"
    },
    {
      "time": 1675358040000,
      "state": "41.0"
    },
    {
      "time": 1675358100000,
      "state": "41.0"
    },
    {
      "time": 1675358160000,
      "state": "41.0"
    },
    {
      "time": 1675358220000,
      "state": "41.0"
    },
    {
      "time": 1675358280000,
      "state": "41.0"
    },
    {
      "time": 1675358340000,
      "state": "41.0"
    },
    {
      "time": 1675358400000,
      "state": "41.0"
    },
    {
      "time": 1675358460000,
      "state": "41.0"
    },
    {
      "time": 1675358520000,
      "state": "41.0"
    },
    {
      "time": 1675358580000,
      "state": "41.0"
    },
    {
      "time": 1675358640000,
      "state": "41.0"
    },
    {
      "time": 1675358700000,
      "state": "41.0"
    },
    {
      "time": 1675358760000,
      "state": "41.0"
    },
    {
      "time": 1675358820000,
      "state": "41.0"
    },
    {
      "time": 1675358880000,
      "state": "41.0"
    },
    {
      "time": 1675358940000,
      "state": "41.0"
    },
    {
      "time": 1675359000000,
      "state": "41.0"
    },
    {
      "time": 1675359060000,
      "state": "41.0"
    },
    {
      "time": 1675359120000,
      "state": "41.0"
    },
    {
      "time": 1675359180000,
      "state": "41.0"
    },
    {
      "time": 1675359240000,
      "state": "41.0"
    },
    {
      "time": 1675359300000,
      "state": "41.0"
    },
    {
      "time": 1675359360000,
      "state": "41.0"
    },
    {
      "time": 1675359420000,
      "state": "41.0"
    },
    {
      "time": 1675359480000,
      "state": "41.0"
    },
    {
      "time": 1675359540000,
      "state": "41.0"
    },
    {
      "time": 1675359600000,
      "state": "41.0"
    },
    {
      "time": 1675359660000,
      "state": "41.0"
    },
    {
      "time": 1675359720000,
      "state": "41.0"
    },
    {
      "time": 1675359780000,
      "state": "41.0"
    },
    {
      "time": 1675359840000,
      "state": "41.0"
    },
    {
      "time": 1675359900000,
      "state": "41.0"
    },
    {
      "time": 1675359960000,
      "state": "41.0"
    },
    {
      "time": 1675360020000,
      "state": "41.0"
    },
    {
      "time": 1675360080000,
      "state": "41.0"
    },
    {
      "time": 1675360140000,
      "state": "41.0"
    },
    {
      "time": 1675360200000,
      "state": "41.0"
    },
    {
      "time": 1675360260000,
      "state": "41.0"
    },
    {
      "time": 1675360320000,
      "state": "41.0"
    },
    {
      "time": 1675360380000,
      "state": "41.0"
    },
    {
      "time": 1675360440000,
      "state": "41.0"
    },
    {
      "time": 1675360500000,
      "state": "41.0"
    },
    {
      "time": 1675360560000,
      "state": "41.0"
    },
    {
      "time": 1675360620000,
      "state": "41.0"
    },
    {
      "time": 1675360680000,
      "state": "41.0"
    },
    {
      "time": 1675360740000,
      "state": "41.0"
    },
    {
      "time": 1675360800000,
      "state": "41.0"
    },
    {
      "time": 1675360860000,
      "state": "41.0"
    },
    {
      "time": 1675360920000,
      "state": "41.0"
    },
    {
      "time": 1675360980000,
      "state": "41.0"
    },
    {
      "time": 1675361040000,
      "state": "41.0"
    },
    {
      "time": 1675361100000,
      "state": "41.0"
    },
    {
      "time": 1675361160000,
      "state": "41.0"
    },
    {
      "time": 1675361220000,
      "state": "41.0"
    },
    {
      "time": 1675361280000,
      "state": "41.0"
    },
    {
      "time": 1675361340000,
      "state": "41.0"
    },
    {
      "time": 1675361400000,
      "state": "41.0"
    },
    {
      "time": 1675361460000,
      "state": "41.0"
    },
    {
      "time": 1675361520000,
      "state": "41.0"
    },
    {
      "time": 1675361580000,
      "state": "41.0"
    },
    {
      "time": 1675361640000,
      "state": "41.0"
    },
    {
      "time": 1675361700000,
      "state": "41.0"
    },
    {
      "time": 1675361760000,
      "state": "41.0"
    },
    {
      "time": 1675361820000,
      "state": "41.0"
    },
    {
      "time": 1675361880000,
      "state": "41.0"
    },
    {
      "time": 1675361940000,
      "state": "41.0"
    },
    {
      "time": 1675362000000,
      "state": "41.0"
    },
    {
      "time": 1675362060000,
      "state": "41.0"
    },
    {
      "time": 1675362120000,
      "state": "41.0"
    },
    {
      "time": 1675362180000,
      "state": "41.0"
    },
    {
      "time": 1675362240000,
      "state": "41.0"
    },
    {
      "time": 1675362300000,
      "state": "41.0"
    },
    {
      "time": 1675362360000,
      "state": "41.0"
    },
    {
      "time": 1675362420000,
      "state": "7.7272448979591815"
    },

Assuming that we don’t need the everyMinute strategy and the REST API only returns values that OH has directly persisted itself, eliminating the everyMinute would mean the Persistence actions would work closer to what one would expect.

Though the carts will look bad so maybe that’s why everyMinute is in the default strategy?

Good point!!

Stupid question, for what timeframe did you querry? The archive one of the default setups does cover one hour only!

I don’t get what you are trying to say in the second part of this sentence. Although the docs state so, IMHO OH is only providing an item state to be persisted by a persistence service.

Why would the cHarts look bad?

1 day, but I only looked at the first 100 or so entries.

If my Item updated two minutes ago and I have an everyUpdate strategy what do I get with lastUpdate?

If rrd4j only returns those values that OH told it to save, lastUpdate should be two minutes ago. If rrd4j returns the values that it’s saved every 20 seconds, lastUpdate will be 20 seconds ago. That’s what I’m trying to figure out. If the former, rrd4j works like any other database (up to a certain amount of time into the past). If not we need to document that persistence calls like lastUpdate won’t return what you expect.

It’s the age old problem. If you have an entry every so often even if the value doesn’t change, you get a nice stair step chart and a straight horizontal line where the data isn’t changing.

If you only record the change, you get a spiky chart with a diagonal line between points meaning that the actual line that makes up the chart doesn’t actually reflect the state of the Item at that time.

If you ask for a day rrd4j will give you the answers from the archive that covers a day. You would need to ask for the last hour to get data from archive one.

Good question, frankly I don’t know! I/We would need to look into that one. But I have to delay that one…

Now I understand, however that is not the case! Using this verbose samplerate a value is stored every 10 seconds. IMHO that does prevent a “spiky chart” even in the higher archives.

Right, but only if those every 10 seconds are returned.

Regarding the lastUpdate I did some trials.
Although I’m using a default datasource and the REST API call shows a value for each 10 second step, lastUpdate does show whole minutes only! At least when writing it ito a log line.
Rechecking the lastUpdate by a rule in small timesteps does show the same timestamp ( whole minute!) for several minutes. Trying to recheck with the REST API I found “holes” in the dara structure, for several minutes NO data was logged at. Can’t say if that was caused by doing a lot of lastUpdate querries or the like. I have to figure out what is really happening.
To be continued.

A value for each 10 second would be returned when requesting a timeframe within archive one. Requesting larger timeframes would be returned with data from the first matching archive. As long as a resolution high enough to prevent those “spikes” was configured all is well. The used resolution ( i.e. the relation between time covered by the archive and its steps) in the default config should ensure that. Opposed to that the use of a single value per day for an archive covering a month would probably not.

Then there is no reason to have an everyMinute strategy be the default for rrd4j. But also it means the add-on docs need a bunch of additions to explain how these queries will work because it’s definitely not clear to me reading the add-on docs as they currently exist. There’s a good set of high level info but important details like this, which impact how the persistence actions work (or not) is missing.

As a user of an unsupervised system susceptible to uncontrolled powerloss who, against the recommendations, has turned Zram off for the persistence directory, I am interested in getting nice staircase charts down to hour timeframe, while minimising SD writes.

Sometimes sensors report more than one value per minute, and sometimes less or even much less. I would like to understand what configuration would be required, but I still don’t.

I think others users may also find that info useful, if that knowledge also could make itself into the docs.

I’m presently trying to find the reason for the observed “holes” in te database. During hose “holes” the item state was no changed. That might be a good reason for keeping the “everyMinute” strategy. Still looking…

1 Like