I’ve just upgraded to openHAB version: v5.0 and just getting some odd values when I get the persistence totals. I’m getting some inconsistent output when I query history for this value vs SQL/db look up.
ie using JS persistence.sumSince I would expect this would return the sum of all from today same as from DB query.
Has anything changed with Javascript persistance?
what is the best way of getting midnight?
Javascript code:
Energy_Mains_energy_interval = items.getItem("Energy_Mains_Total_interval")
var daySoFar = Energy_Mains_energy_interval.persistence.sumSince(time.toZDT("00:00")).numericState //sum since midnight
console.log("Energy: this daily= " + daySoFar);
output:
Energy: this daily= 124353240
SQL code:
select sum(value) from energy_mains_total_interval_46
where time >= '2025-07-27 00:00:00';
nope TZ look ok , this only started after the upgrade to OH 5.0 , is this a bug?
var midnight=time.toZDT("00:00");
var daySoFar = Energy_Mains_energy_interval.persistence.sumSince(midnight).numericState //sum since midnight
console.log("Energy: this daily= " + daySoFar + ' md: '+ midnight);
output
Energy: this daily= 82092600 md: 2025-07-28T00:00+10:00[Australia/Canberra]
I just output the state and its returning the following:
Energy: this daily= 3164219.9999999999812800000 Ws
what is “Ws” Watt Second? and how did was this unit type assigned ?
items:
Energy_Mains_Total_interval:
type: Number
dimension: Energy
label: Energy Mains Total watt minutes
icon: energy
groups:
- Power_meter
tags:
- Sensor
If I divide this value by 3,600,000 I get pretty close to what I expect, how is persistence returning this unit
Persistence saves the state of the Item as a number. During queries and restoreOnStartup it assumes the unit of the Item is the unit of the result.
If no unit metadata is on the Item, the system default is assumed. I thought the default for Energy was Wh but it might be Ws.
Regardless, the Item is using Ws as the unit so the Item’s state is always in Ws and therefore all the values in persistence are in Ws also.
Note that there is a UoM value saved in the Persisted state methods. So you could get the result and convert them to Wh with
var daySoFar = Energy_Mains_energy_interval.persistence.sumSince(midnight).quantityState.toUnit("Wh");
Or you can set the unit metadata on the Item to “Wh” and the Item’s state and therefore the persistence calls will all be in “Wh” by default. However, the changes are not retroactive. The values already stored in persistence will not be adjusted. Furthermore, those old values will be assumed to be in Wh so your calculations with be way off.
The only part I don’t quite understand is how the SQL query is returning the correct value in Wh. I don’t use SQL so it might be that those databases, which can be more flexible than rrd4j for example, do some unit conversions for you. I don’t know.
I had similar issues with persistence actions, I have an Item connected to the Tibber binding for getting spot prices, and wanted to calculate the standard deviation for the last 14 days. The Item has unit set to SEK/kWh, but when using the deviationSince action returns the value as “SEK/(W·s)”. I thought this was due to the calculations done to get the deviation (and it is only when logging the value it’s noticeable, when updating an Item with correct unit, everything works as it should), so didn’t pay it much mind. But when I saw this thread I got curious and tested different actions:
The since deviation is based on variance, it’s as I thought not surprising that the unit gets mangled and uses the system defalut. Not sure how the riemannSumBetween gets calculated, but should be similar to an integral, in which case it’s logical that the temporal dimension is removed.
But I would expect at least the regular sumBetween to return the correct unit.
Again, all values are correct, and if using the value to update an Item that has the correct unit set there’s no issue. But if it’s expected of the persistence actions to use the unit metadata (provided the result is in the same dimension as the Item) it seems to be a bug somewhere.
I wouldn’t call it a bug, as the calculated results are correct. There may be room for improvement. All calculations are done in the base unit. It may not matter for most units, but it does for things like temperature where the base unit is K. Also, persistence services could return individual values with different units if they store the unit with persisted values (the typical ones don’t, just assume the configured unit).
What could be an option is to convert the result of the calculation to the (adjusted for dimensional calculation) configured unit in code.
There was never a guarantee the calculation would return a result in the base unit. The issue stems from the fact that you just drop the unit and make the assumption it is in the base unit. If you get a QuantityType back, you should convert it to the unit you want before dropping the unit, or never drop the unit at all and keep calculating with Quantities. And that’s a dangerous assumption. It might have worked in some cases in the past, but it would have created totally wrong results for dimensions like temperature, or for persistence services that had values stored with different units.
I do agree the outcome has probably changed and I am not against adjusting it, but there will always be cases where it is unexpected if you don’t consider the unit.
Variance and riemannSum do some additional calculations on the unit (square and multiply with second respectively), so in those cases it might be better to use the system default, but for deviation and sum I see no reason not to use the configured unit?
Is there a particular reason, or should I submit a PR to change them to the top variant?
@pacive Deviation is calculated internally as the squareroot of variance. It also has a time factor in it. So as variance is already using system unit in its calculation, it keeps on working with that.
If you don’t use system unit for sum, you run into trouble with temperatures. I honestly think the old sum methods are pretty stupid, as the time dimension is completely ignored (that’s why I added Riemann Sums). But still, to get mathematically and physically meaningful and correct results, you have to do the calculation in system unit.
In many cases, it would be an improvement to convert the result to the base unit after the calculation and before returning it in the method. That would most likely pretty much align the result with what it was before. I am open for such a change in the code.
I believe all methods where it returns base unit at the moment are methods where the calculation results in the same quantity (apart from unit conversion) whatever the unit is used for calculation. Where there could be a different (and I am specifically referring to temperatures), it uses system unit.
That is closer to what the caller expects I think. They set the unit on their Item. Everything else goes out of it’s way to keep the values in that unit, setting the expectation that in the rule, one will always get the value in that unit.
If it’s feasible to convert the result of the calculation back to the expected unit I think that would be better.
So persistence can convert everything to K (for temperatures) to do the calculation and then back to degrees C or F before returning.
