ECMA-2021 / rrd4j: averageSince only for "longer" time interval?

running OH3.3-release on my Docker on Synology.

I’d like to calculate an “averageSince” of an Number-item with rrd4j standard persistence and get a regular “null” - until I expand the time interval of about 45 seconds?
timezone on my Synology and docker is correct and rrd4j should deliver also for shorter time intervals, shouldn’t it?


var interval = time.ZonedDateTime.now().minusSeconds(20);
console.info("20sec: " + interval)
var grundlast = items.getItem("KOS_Verbrauch").history.averageSince(interval);

console.info("Grundlast: ",  grundlast);
if (grundlast != null) {
  items.getItem("EMS_Grundlast").sendCommand(grundlast.toString());
} else { 
  console.info("leider null"); 
}

leads to:

2022-10-20 14:39:42.726 [INFO ] [b.automation.script.ui.EMS_Grundlast] - 20sec: 2022-10-20T14:39:22.721+02:00[SYSTEM]
2022-10-20 14:39:42.888 [INFO ] [b.automation.script.ui.EMS_Grundlast] - Grundlast:  null
2022-10-20 14:39:42.889 [INFO ] [b.automation.script.ui.EMS_Grundlast] - leider null

if I expand the time interval to say 45 seconds, it works:

2022-10-20 14:40:41.047 [INFO ] [b.automation.script.ui.EMS_Grundlast] - 45sec: 2022-10-20T14:39:56.041+02:00[SYSTEM]
2022-10-20 14:40:41.402 [INFO ] [b.automation.script.ui.EMS_Grundlast] - Grundlast:  458.2753273104981

--

2022-10-20 14:40:41.404 [INFO ] [openhab.event.ItemCommandEvent      ] - Item 'EMS_Grundlast' received command 458.2753273104981
2022-10-20 14:40:41.405 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'EMS_Grundlast' changed from 456.0 to 458.2753273104981

That’s probably expected behavior unless this Item is changing at least twice per 20 seconds. If there’s only one result over the time period, averageSince returns null because you can’t calculate an average with only one value.

I suspect that you don’t have at least two values in the database until you expand the time to 45 seconds.

Unrelated note, there are some alternative ways to generate interval.

var interval = time.toZDT().minusSeconds(20);
var interval = time.toZDT(-20000);
var interval = time.toZDT('PT-20s');
var interval = time.toZDT(time.Duration.ofSeconds(-20));

toZDT() will take almost anything you can think of that represents a time or a duration and converts it to a ZonedDateTime. See Working with Date Times in JS Scripting (ECMAScript 11).

1 Like

the item I’m averaging on is my current energy consumption and it comes every few seconds or so, that should not be the thing here. could be rrd4j itself, that accept/stores information differently?

2022-10-20 15:35:15.898 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'KOS_Verbrauch' changed from 407 to 423
2022-10-20 15:35:21.730 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'KOS_Verbrauch' changed from 423 to 403
2022-10-20 15:35:27.552 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'KOS_Verbrauch' changed from 403 to 400
2022-10-20 15:35:33.425 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'KOS_Verbrauch' changed from 400 to 414
2022-10-20 15:35:39.181 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'KOS_Verbrauch' changed from 414 to 403
2022-10-20 15:35:44.898 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'KOS_Verbrauch' changed from 403 to 413
2022-10-20 15:35:50.749 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'KOS_Verbrauch' changed from 413 to 412
2022-10-20 15:35:56.552 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'KOS_Verbrauch' changed from 412 to 409
2022-10-20 15:36:02.284 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'KOS_Verbrauch' changed from 409 to 417

strange thing: sometimes 45secs also return ‘null’ - I’ll let it check via cron: " At 59 seconds past the minute", notice the gap of sometims >2secs?

2022-10-20 15:31:00.742 [INFO ] [tomation.script.ui.EMS_CalcGrundlast] - Ermittle die Grundlast:  null
2022-10-20 15:32:01.585 [INFO ] [tomation.script.ui.EMS_CalcGrundlast] - Ermittle die Grundlast:  394.0652998405980
2022-10-20 15:33:00.160 [INFO ] [tomation.script.ui.EMS_CalcGrundlast] - Ermittle die Grundlast:  null
2022-10-20 15:34:00.081 [INFO ] [tomation.script.ui.EMS_CalcGrundlast] - Ermittle die Grundlast:  393.1388888888890
2022-10-20 15:35:02.091 [INFO ] [tomation.script.ui.EMS_CalcGrundlast] - Ermittle die Grundlast:  null
2022-10-20 15:35:59.862 [INFO ] [tomation.script.ui.EMS_CalcGrundlast] - Ermittle die Grundlast:  null
2022-10-20 15:37:00.680 [INFO ] [tomation.script.ui.EMS_CalcGrundlast] - Ermittle die Grundlast:  429.8198778657557
2022-10-20 15:38:00.043 [INFO ] [tomation.script.ui.EMS_CalcGrundlast] - Ermittle die Grundlast:  null

:heart_eyes:

Depends on whether you have an rrd4j.persist and what’s in it. Without overriding the default strategy, rrd4j will save on everyChange.

You can query the database through the OH API Explorer and see if it’s storing values as expected.

I didn’t change the default strategy. And the values get persisted pretty good - but:

but: there’s a gap from the current time until the last chart point. But I’ll notice the same on all items/charts with rrd4j-persistence.

If I change to MySQL-persisted items/charts, I’ll get the following:

…and I tried the above script on my main openHAB with MySQL persistence on default and there’s the averageSince is always != null? Guess, either the rrd4j isn’t the best solution for that or something else is interfering…

That rrd4j chart looks choppy. Query the db through the API explorer to verify that you are getting the data you think you are. It loks like you are at most getting a value every 20 to 30 seconds compared to the squiggles you see in the MySQL chart showing a value every second or so.

Perhaps there is something about how rrd4j works that limits how many of these values it can store. The docs mention that for plain numbers it has a granularity of 10 seconds for the last hour. I’m not exactly sure how to interpret that (@opus, you’re the resident rrd4j expert, can you give details) but based on looking at your chart and some of my own I’m wondering if it won’t save more than one value every ten seconds now matter how fast the Item is changing.

1 Like

From what I’m seeing, there’s only a value every 120seconds. If that falls within the timeframe of the script - it’ll show an averageSince value, I guess…

{
  "name": "KOS_Verbrauch",
  "datapoints": "290",
  "data": [
...
    {
      "time": 1666275780000,
      "state": "1948.0"
    },
    {
      "time": 1666275840000,
      "state": "496.0"
    },
    {
      "time": 1666275900000,
      "state": "496.0"
    },
    {
      "time": 1666275960000,
      "state": "496.0"
    },
    {
      "time": 1666276020000,
      "state": "500.1666666666667"
    },
    {
      "time": 1666276080000,
      "state": "501.0"
    }
  ]
}

It’s looking like rrd4j may not be suitable for your use case. You might need to use MySQL or InfluxDB or something like that with some sort of cleanup to delete values that get too old (so your DB doesn’t grow too large) for your rule. Or keep a running average as a variable in your rules. You can use the cache to save it and make it available across your rules.

1 Like

That’s a great idea and something new to learn! :wink:

(Normally I use MySQL-persistence, so I’m not in trouble; but first, I’d like to have everything in place on my new Docker-OH3, before I switch my main OH3 to the Docker; too many changes in item names and stuff, so the DB-persistence will get spammed with dead item-tables and stuff… #ocdwontallowthis)

I feel really honored the be called something like that by “the expert”.

Despite the honors above I do not have a real deep knowledge of the actual code!

IMHO the usage of data in relation of openHAB to rrd4j is twofold.
openHAB does provide data to be persisted as set in the .persist file by the strategy (everyMinute, everyChange…). Note the: “to be persisted!”
rrd4j does use data in fixed timesteps by design, the steps are defined in the archive settings ( set in the rrd4j.cfg or by the defaults as the SampleInterval). In my understanding in case of a change of state within a SampleIntervall ( default 10 sec, the “granularity”) only the last value would be stored by rrd4j. In order to store in smaller timesteps a custon rrd4j.cfg with a SampleIntervall of 1 sec might be created. I never tried such and I can’t say if it would work nor what the system load would be.

My guess on the gap in the chart would be the used timestep of the displayed archive. The posted chart shows data for a hour. That would be archive one of the default_numeric archive setting. This archive uses one SampleIntervall as its timestep, hence 10 seconds. As the posted chart shows a gap of a minute I’m proven wrong. Actually I have no other clue :slightly_frowning_face:

2 Likes

What timeframe did you request? You need to querry for the last hour or less in order to get data from archive 1!

He was querying for the last 20 seconds and getting null which is an indication that there is one or zero values for the past 20 seconds.

I don’t think either @binderth or I are overly concerned about the gap at the end of the chart as much as we are about the failure of averageSince to return a meaningful result unless the time is 45 seconds or later, even though the Item is changing every few seconds.

This would explain what I’m seeing here I think.

I must add: I activated persistence today around 12:00.
I requested a whole day.

Despite that I do think, rrd4j is somewhat cool for charts (especially for multi-line charts), as all items get persisted on the same time in rrd4j - as in MySQL the items get persisted onChange (and perhaps in a strategy again). Which makes it possible for rrd4j-charts to have a neat tool-tip and nearly impossible for DB-based-charts to have all values on the same y-axis to show up in the tool-tip in parallel.
So for historic reasons (e.g. averageSince) DB-based persistence has advantages and for graphical purposes (e.g. OH3-charts) rrd4j-based persistence has advantages. correct?

1 Like

I’m sorry, my question should have been phrased " What timeframe was used for the API Explorer call posted". The posted output does show a minute timegap as if a whole day was called for.

If I request the last 2minutes starting with the actual time backwards, it comes back as:

{
  "name": "KOS_Verbrauch",
  "datapoints": "0",
  "data": []
}

if I do that for 2 minutes and after 2 minutes (e.g. 18:33 - 18:35 and it’s 18:37) same.
only if it’s 3 minutes back I’ll get:

{
  "name": "KOS_Verbrauch",
  "datapoints": "13",
  "data": [
    {
      "time": 1666283580000,
      "state": "776.1666666666666"
    },
    {
      "time": 1666283590000,
      "state": "776.1666666666666"
    },
    {
      "time": 1666283600000,
      "state": "776.1666666666666"
    },
    {
      "time": 1666283610000,
      "state": "776.1666666666666"
    },
    {
      "time": 1666283620000,
      "state": "776.1666666666666"
    },
    {
      "time": 1666283630000,
      "state": "776.1666666666666"
    },
    {
      "time": 1666283640000,
      "state": "776.1666666666666"
    },
    {
      "time": 1666283650000,
      "state": "776.1666666666666"
    },
    {
      "time": 1666283660000,
      "state": "776.1666666666666"
    },
    {
      "time": 1666283670000,
      "state": "776.1666666666666"
    },
    {
      "time": 1666283680000,
      "state": "776.1666666666666"
    },
    {
      "time": 1666283690000,
      "state": "776.1666666666666"
    },
    {
      "time": 1666283700000,
      "state": "776.1666666666666"
    }
  ]
}

So it looks like rrd4j needs “some” time before it can spit out thd data that it is storing.

1 Like