averageSince not working as expected

  • Platform information:
    • Hardware: 64bit x86 (Fujitsu Primergy)
    • OS: Linux 4.19.0-20-amd64
    • Java Runtime Environment: openjdk 11.0.17 2022-10-18 LTS OpenJDK Runtime Environment Zulu11.60+19-CA (build 11.0.17+8-LTS) OpenJDK 64-Bit Server VM Zulu11.60+19-CA (build 11.0.17+8-LTS, mixed mode)

My OH is 3.40 release build.

I don’t know whether I understood the function of averageSince correctly. I understood that averageSince corrects the averaging over time, that is: if I have 1 minute a value of 10.0 and 59 minutes a value of 10.0 it is supposed to return 10 * 1/60 or 0.166666667 for averageSince(now.minusHours(1)).

I use influxdb as the default persistence and I persist on every change on default. My results are not as expected. I only get a value of 5.0. I generated the data with a time cron activated rule and fed these into two number items. One was persisted as mentioned above, the other one also every minute. If I operate averageSince on the item being persisted every minute I get the expected result.

rule "switch"

when
    Time cron "0 0 0/1 * * ?"   // stündlicher Aufruf volle Stunde
then
    szTest.sendCommand(10.0)
    szTest2.sendCommand(10.0)
    Thread::sleep(60000)        // 1 Minute warten
    szTest.sendCommand(0.0)
    szTest2.sendCommand(0.0)
    logInfo ('rule','Test-Mittelwerte: {} - {}', szTest.averageSince(now.minusHours(24)), szTest2.averageSince(now.minusHours(24)))
end

Ergebnis

2023-01-08 14:01:00.780 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 5.001833241671250
2023-01-08 15:01:00.722 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.2459422204432034
2023-01-08 16:01:00.721 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.2066317591430457
2023-01-08 17:01:00.722 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.1933840364753228

This lets me conclude that averageSince is not time corrected at all.

Did I miss something ?

hello
did you read this ?
https://www.openhab.org/docs/configuration/persistence.html#time-weighted-averages

Sure. It says

Gets the average value of the State of a persisted Item since a certain point in time. This method uses a time-weighted average calculation (see example below)

My results do not confirm that IMHO.

Possibly. Commands and Updates to Items are processed in parallel in the background. It is almost certainly the case that your two commands to szTest and szTest2 after the sleep have not yet been fully processed and the new states saved to the database before you get to that logInfo that calls averageSince.

Okay, but that should affect both items then. Where does the difference come from ? I have now complete 24 h results, and these show IMHO that the averageSince does NOT do any time based calculations (or it does it in a way I don’t understand).

2023-01-08 14:01:00.780 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 5.001833241671250
2023-01-08 15:01:00.722 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.2459422204432034
2023-01-08 16:01:00.721 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.2066317591430457
2023-01-08 17:01:00.722 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.1933840364753228
2023-01-08 18:01:00.724 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.1867323709231003
2023-01-08 19:01:00.721 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.1827320173307293
2023-01-08 20:01:00.722 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.1800620915704080
2023-01-08 21:01:00.723 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.1781527957344010
2023-01-08 22:01:00.740 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.1767231995411775
2023-01-08 23:01:00.722 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.1756058120246353
2023-01-09 00:01:00.725 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.1747139236029958
2023-01-09 01:01:00.723 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.1739842257979877
2023-01-09 02:01:00.724 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.1733753857812170
2023-01-09 03:01:00.725 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.1728599869551712
2023-01-09 04:01:00.724 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.1724181359128261
2023-01-09 05:01:00.724 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.1720352223060818
2023-01-09 06:01:00.724 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.1717001231374701
2023-01-09 07:01:00.725 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.1714043244045849
2023-01-09 08:01:00.725 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.1711413644673289
2023-01-09 09:01:00.728 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.1709063449373488
2023-01-09 10:01:00.732 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.1706948748441489
2023-01-09 11:01:00.737 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.1705035265680817
2023-01-09 12:01:00.799 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.1703331598873389
2023-01-09 13:01:00.727 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.1701696658035637
2023-01-09 14:01:00.727 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.1666682696768721
2023-01-09 15:01:00.727 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 5.0 - 0.1666682503871039

Not necessarily. It can take some milliseconds to perform an averageSince. The data may not be there for the first Item but the time it takes to calculate the average on that first Item is enough that the data is there for the second Item once you get to the call to that averageSince.

Add a 100-500 msec sleep before your logInfo to ensure that both Items have updated and saved to persistence before trying to pull the averageSince.

I’m not saying whether or not the time based calculations are happening (though the code appears to be there to do it). I’m just pointing out that you are currently likely seeing problems caused by timing.

Okay - will do that and report back.

I added 500 ms delay before averageSince. These are the new results:

2023-01-09 17:01:01.338 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 4.999969708163012 - 0.1667826029348438
2023-01-09 18:01:01.282 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 4.999969768487439 - 0.1667826531989327
2023-01-09 19:01:01.282 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 4.999969768487074 - 0.1667827111091942
2023-01-09 20:01:01.291 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 4.999969708144733 - 0.1667827535659630
2023-01-09 21:01:01.294 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 4.999969647803491 - 0.1667829195699779
2023-01-09 22:01:01.299 [INFO ] [org.openhab.core.model.script.rule  ] - Test-Mittelwerte: 4.999969708145464 - 0.1667826802022226

I don’t see a big difference. The first value is bad (persisted on change), the second is good (persisted on change and every minute).

Please guide me what I am to do next. I am not expecting a solution for this but I am willing to contribute. I. e. figuring out how to file a bug report or even help to find a solution. But before I start to “mess things up” I would ask if somebody can check my conclusions. Thanks a lot.

To file an issue see How to file an Issue.

I think the relevant code for this is in the openhab-core repo.

Assuming your interpretation is correct and it’s calculating incorrectly than the next step is to file an issue. Either the docs are wrong or the code is wrong.

What is the current state of this topic? Did you file an issue?
I have a similar problem and am getting a different averageSince, than expected.

I did not know how to do that - so no, I didn’t.

All right, I had a look at the code of averageSince.
The problem is that the value is weighted with the timespan to the previous HistoricItem instead of the timespan to the next HistoricItem.

If you store the values every minute, all timespans are the same and it does not really matter. However, if you store it only every change, it might make a huge difference.

@mschlee: Might this explain your observations too?

I will try to fix it and create a pull request.

That’s exactly what I observed. Thanks for taking care of this.

you could use rrd4j in parallel (doesn’t hurt that much - I’ve got >1500 items). That’s an every-minute default persistence and should work.

Thanks for the hint. My first post here explained that I can individually configure influxdb to use a 1 minute persistence for items which are not updated periodically - so I found a way to get what I want. The intention of this post was to show that averageSince does not work as expected - which has been confirmed now by at least one other user. I hope that this helps others.

2 Likes

All right, thank you.

I have created the following pull request to fix it: #3439

2 Likes